[DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Nicolai Stange]

Hi all,

first of all, my apologies for the patch bomb following up in reply to this
mail here -- it's not meant to receive any serious review at all, but only
to support the discussion I'm hoping to get going.

As some of you might already be aware of, all new submissions for FIPS
certification will be required to comply with NIST SP800-90B from Nov 7th
on ([1], sec. 7.18 "Entropy Estimation and Compliance with SP 800-90B").
For reference: broadly speaking, NIST SP800-90B is about noise sources,
SP800-90A about the DRBG algorithms stacked on top and SP800-90C about how
everything is supposed to be glued together. The main requirements from
SP800-90B are
- no correlations between different noise sources,
- to continuously run certain health tests on a noise source's output and
- to provide an interface enabling access to the raw noise samples for
  validation purposes.

To my knowledge, all SP800-90B compliant noise sources available on Linux
today are either based on the Jitter RNG one way or another or on
architectural RNGs like e.g. x86's RDSEED or arm64's RNDRRS. Currently,
there's an in-kernel Jitter RNG implementation getting registered (c.f.
crypto/drbg.c, (*)) with the Crypto RNG API, which is also accessible from
userspace via AF_ALG. The userspace haveged ([2]) or jitterentropy
integrations ([3]) are worth mentioning in this context, too. So in
summary, I think that for the in-kernel entropy consumers falling under the
scope of FIPS, the currently only way to stay compliant would be to draw it
from said Crypto API RNG. For userspace applications there's the additional
option to invoke haveged and alike.

OTOH, CPU jitter based techniques are not uncontroversial ([4]). In any
case, it would certainly be a good idea to mix (xor or whatever) any jitter
output with entropy obtained from /dev/random (**). If I'm not mistaken,
the mentioned Crypto API RNG implementation (crypto/drbg.c) follows exactly
this approach, but doesn't enforce it yet: there's no
wait_for_random_bytes() and early DRBG invocations could in principle run
on seeds dominated entirely by jitterentropy. However, this can probably
get sorted quite easily and thus, one reasonable way towards maintaining
FIPS resp. SP800-90 compliance would be to
- make crypto/drbg.c invoke wait_for_random_bytes(),
- make all relevant in-kernel consumers to draw their random numbers from
  the Crypto RNG API, if not already the case and
- convert all relevant userspace to use a SP800-90B conforming Jitter RNG
  style noise source for compliance reasons, either by invoking the
  kernel's Crypto RNG API or by diffent means, and mix that with
  /dev/random.

Even though this would probably be feasible, I'm not sure that giving up on
/dev/random being the primary, well established source of randomness in
favor of each and every userspace crypto library rolling its own entropy
collection scheme is necessarily the best solution (it might very well be
though).

An obvious alternative would be to make /dev/random conform to SP800-90B.
Stephan Müller posted his "LRNG" patchset ([5]), in which he proposed to
introduce a second, independent implementation aiming at SP800-90[A-C]
conformance. However, it's in the 35th iteration now and my impression is
that there's hardly any discussion happening around this for quite a while
now. I haven't followed the earlier development, but I can imagine several
reasons for that:
- people are not really interested in FIPS or even questioning the whole
  concept in the first place (c.f. Theodore Ts'o remarks on this topic
  at [6]),
- potential reviewers got merely discouraged by the diffstat or
- people dislike the approach of having two competing implementations for
  what is basically the same functionality in the kernel.

In either case, I figured it might perhaps help further discussion to
provide at least a rough idea of how bad the existing /dev/random
implementation would get cluttered when worked towards SP800-90B
compliance. So I implemented the required health tests for the interrupt
noise source -- the resulting patches can be found in reply to this mail.
I'd like to stress(!) that this should really only be considered a first
step and that there would still be a long way towards a complete solution;
known open items are listed below. Also, I'm fully aware that making those
continuous health tests block the best effort primary_crng reseeds upon
failure is a ridiculous thing to do -- that's again meant for demonstration
purposes only, c.f. the commit log from the next to last patch. Anyway,
those of you who are interested in some more details beyond the mere
diffstat can find them after the list of references below.

In summary, I can imagine three feasible ways towards SP800-90 compliance:
1.) Put the burden on consumers. For in-kernel users this would mean
    conversion to the Jitter backed Crypto RNG API, in case that hasn't
    happened yet. Userspace is free to use any approved Jitter based
    mechanism for compliance reasons, but is encouraged to mix that with
    /dev/random.
2.) Merge Stephan's LRNG. Users/distros would have to decide between either
    of the two competing implementations at kernel config time.
3.) Develop the existing /dev/random towards compliance, ideally w/o
    affecting !fips_enabled users too much. This would likely require some
    redundancies as well as some atrocities imposed by the specs.

I'm looking forward to hearing your opinions and suggestions! In case you
happen to know of anybody who's not on CC but might potentially be
interested in FIPS, I'd highly appreciate it if you could point him/her to
this thread. The usual suspects are probably (enterprise?) distro folks,
but there might be others I haven't thought of.

Many thanks for your time!

Nicolai


(*) That's an oversimplification for the sake of brevity: actually
    SP800-90A DRBGs stacked on top of the SP800-90B conforming
    jitterentropy source get registered with the Crypto API.
(**) "/dev/random" is used as a synonym for everything related to
     drivers/char/random.c throughout this mail.

[1] https://csrc.nist.gov/csrc/media/projects/cryptographic-module-validation-program/documents/fips140-2/fips1402ig.pdf
[2] http://www.issihosts.com/haveged/
[3] http://www.chronox.de/jent/doc/CPU-Jitter-NPTRNG.html
    c.f. appendices C-E
[4] https://lwn.net/Articles/642166/
[5] https://lkml.kernel.org/r/5667034.lOV4Wx5bFT@positron.chronox.de
[6] https://lkml.kernel.org/r/20170919133959.5fgtioyonlsdyjf5@thunk.org
    https://lkml.kernel.org/r/20170920011642.cczekznqebf2zq5u@thunk.org
[7] https://lkml.kernel.org/r/aef70b42-763f-0697-f12e-1b8b1be13b07@gmail.com


As promised above, some more details on the RFC series sent alongside
follow. The primary goal was to implement that health test functionality as
required by SP800-90B for the existing drivers/char/random.c without
affecting !fips_enabled users in any way. As outlined below, I failed quite
miserably as far as performance is concerned, but that shouldn't be
something which cannot get rectified. Kernel version v5.9-rc4 had been used
as a basis. The series can be logically subdivided into the following
parts:
- [1-5]: Preparatory cleanup.
- [6-17]: Implement support for deferring entropy credit dispatch to the
  global balance to long after the corresponding pool mixing operation has
  taken place. Needed for "holding back" entropy until the health tests
  have finished on the latest pending batch of samples.
- [18-21]: Move arch_get_random_{seed_,}long() out of the interrupt path.
  Needed to adhere to how SP800-90C expects multiple noise source to get
  combined, but is also worthwhile on its own from a performance POV.
- [22-23]: Don't award entropy to non-SP800-90B conforming architectural
  RNGs if fips_enabled is set.
- [24]: Move rand_initialize() to after time_init(). A "fix" for what is
  currently a non-issue, but it's a prerequisite for the subsequent patch.
- [25]: Detect cycle counter resolution, subsequently needed for making a
  per-IRQ entropy assessment.
- [26-28]: Follow Stephan's LRNG approach in how much entropy gets
  awarded to what: a lot more than before to add_interrupt_randomness(),
  none to add_{disk,input}_randomness() anymore.
- [29-33]: Introduce empty health test stubs and wire them up to
  add_interrupt_randomness().
- [34-36]: Implement the Adaptive Proportion Test (APT) as specified by
  SP800-90B and squeeze some more statistical power out of it.
- [37]: Implement SP800-90B's Repetition Count Test (RCT).
- [38-40]: Implement the startup tests, which are nothing but the
  continuous tests (APT + RCT) run on a specified amount of samples at
  boot time.
- [41]: Attempt to keep the system going in case the entropy estimate
  had been too optimistic and the health tests keep failing.

As the health tests are run from interrupt context on each sample, a
performance measurement is due. To this end, I configured a Raspberry Pi 2B
(ARMv7 Cortex A7) to disable all peripherals, gated a
19.2 MHz / 2048 ~= 9.3 kHz clock signal to some edge triggered GPIO and
function_graph traced add_interrupt_randomness() for 10 min from a busybox
initramfs. Unfortunately, the results had been a bit disappointing: with
fips_enabled being unset there had been a runtime degradation of ~12.5% w/o
SMP and ~5% w/ SMP resp. on average merely due to the application of the
patches onto the v5.9-rc4 base. However, as the amount of work should not
have changed much and given that struct fast_pool still fits into a single
cacheline, I'm optimistic that this can get rectified by e.g. introducing
a static_key for fips_enabled and perhaps shuffling branches a bit such
that the !fips_enabled code becomes more linear. OTOH, the impact of
enabling the health tests by means of setting fips_enabled had not been so
dramatic: the observed increase in average add_interrupt_randomness()
runtimes had been 6% w/o SMP and 5% w/ SMP respectively.

Apart from those well controlled experiments on a RPi, I also did some
lax benchmarking on my x86 desktop (which has some Intel i9, IIRC).
More specifically, I simply didn't touch the system and ftraced
add_interrupt_randomness() for 15 mins. The number of captured events had
been about 2000 in each configuration. Here the add_interrupt_randomness()
performance improved greatly: from 4.3 us on average w/o the patches down
to 2.0 us with the patches applied and fips_enabled. However, I suppose
this gain was due to the removal of RDSEED from add_interrupt_randomness().
Indeed, when inspecting the distribution of add_interrupt_randomness()
runtimes on plain v5.9-rc4 more closely, it can be seen that there's a
good portion of events (about 1/4th) where add_interrupt_randomness() took
about 10us. So I think that this comparison isn't really a fair one...


To the best of my knowledge, these are the remaining open questions/items
towards full SP800-90[A-C] compliance:
- There's no (debugfs?) interface for accessing raw samples for validation
  purposes yet. That would be doable though.
- try_to_generate_entropy() should probably get wired up to the health
  tests as well. More or less straightfoward to implement, too.
- Diverting fast_pool contents into net_rand_state is not allowed (for a
  related discussion on this topic see [7]).
- I've been told that SP800-90A is not a hard requirement yet, but I
  suppose it will eventually become one. This would mean that the chacha20
  RNG would have to get replaced by something approved for fips_enabled.
- The sequence of fast_pool -> input_pool -> extract_buf() operations
  is to be considered a "non-vetted conditioning component" in SP800-90B
  speak. It would follow that the output can't be estimated as having full
  entropy, but only 0.999 of its length at max. (c.f. sec. 3.1.5.2). This
  could be resolved by running a SP800-90A derivation function at CRNG
  reseeding for fips_enabled. extract_buf(), which is already SHA1 based,
  could perhaps be transformed into such one as well.
- The only mention of combining different noise sources I was able to find
  had been in SP800-90C, sec. 5.3.4 ("Using Multiple Entropy Sources"):
  it clearly states that the outputs have to be combined by concatenation.
  add_hwgenerator_randomness() mixes into the same input_pool as
  add_interrupt_randomness() though and I would expect that this isn't
  allowed, independent of whether the noise source backing the former
  is SP800-90B compliant or not. IIUC, Stephan solved this for his LRNG
  by maintaing a separate pool for the hw generator.
- SP800-90A sets an upper bound on how many bits may be drawn from a
  DRBG/crng before a reseed *must* take place ("reseed_interval"). In
  principle that shouldn't matter much in practice, at least not with
  CONFIG_NUMA: with reseed_interval == 2^32 bits, a single CRNG instance
  would be allowed to hand out only 500MB worth of randomness before
  reseeding, but a (single) numa crng chained to the primary_crng may
  produce as much as 8PB before the latter must eventually get reseeded
  from the input_pool. But AFAICT, a SP800-90A conforming implementation
  would still have to provide provisions for a blocking extract_crng().
- It's entirely unclear to me whether support for "prediction resistance
  requests" is optional. It would be a pity if it weren't, because IIUC
  that would effectively imply a return to the former blocking_pool
  behaviour, which is obviously a no-no.


Nicolai Stange (41):
  random: remove dead code in credit_entropy_bits()
  random: remove dead code for nbits < 0 in credit_entropy_bits()
  random: prune dead assignment to entropy_bits in credit_entropy_bits()
  random: drop 'reserved' parameter from extract_entropy()
  random: don't reset entropy to zero on overflow
  random: factor the exponential approximation in credit_entropy_bits()
    out
  random: let pool_entropy_delta() take nbits in units of
    2^-ENTROPY_SHIFT
  random: introduce __credit_entropy_bits_fast() for hot paths
  random: protect ->entropy_count with the pool spinlock
  random: implement support for delayed entropy dispatching
  random: convert add_timer_randomness() to queued_entropy API
  random: convert add_interrupt_randomness() to queued_entropy API
  random: convert try_to_generate_entropy() to queued_entropy API
  random: drop __credit_entropy_bits_fast()
  random: convert add_hwgenerator_randomness() to queued_entropy API
  random: convert random_ioctl() to queued_entropy API
  random: drop credit_entropy_bits() and credit_entropy_bits_safe()
  random: move arch_get_random_seed() calls in crng_reseed() into own
    loop
  random: reintroduce arch_has_random() + arch_has_random_seed()
  random: provide min_crng_reseed_pool_entropy()
  random: don't invoke arch_get_random_long() from
    add_interrupt_randomness()
  random: introduce arch_has_sp800_90b_random_seed()
  random: don't award entropy to non-SP800-90B arch RNGs in FIPS mode
  init: call time_init() before rand_initialize()
  random: probe cycle counter resolution at initialization
  random: implement support for evaluating larger fast_pool entropies
  random: increase per-IRQ event entropy estimate if in FIPS mode
  random: don't award entropy to disk + input events if in FIPS mode
  random: move definition of struct queued_entropy and related API
    upwards
  random: add a queued_entropy instance to struct fast_pool
  random: introduce struct health_test + health_test_reset()
    placeholders
  random: introduce health test stub and wire it up
  random: make health_test_process() maintain the get_cycles() delta
  random: implement the "Adaptive Proportion" NIST SP800-90B health test
  random: improve the APT's statistical power
  random: optimize the APT's presearch
  random: implement the "Repetition Count" NIST SP800-90B health test
  random: enable NIST SP800-90B startup tests
  random: make the startup tests include muliple APT invocations
  random: trigger startup health test on any failure of the health tests
  random: lower per-IRQ entropy estimate upon health test failure

 arch/arm64/include/asm/archrandom.h   |   33 +-
 arch/powerpc/include/asm/archrandom.h |   17 +-
 arch/s390/include/asm/archrandom.h    |   19 +-
 arch/x86/include/asm/archrandom.h     |   26 +-
 drivers/char/random.c                 | 1141 ++++++++++++++++++++++---
 include/linux/random.h                |   17 +
 init/main.c                           |    2 +-
 7 files changed, 1101 insertions(+), 154 deletions(-)

--
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg), GF: Felix Imendörffer

-- 
2.26.2

[RFC PATCH 01/41] random: remove dead code in credit_entropy_bits()

[Nicolai Stange]

Since commit 90ea1c6436d2 ("random: remove the blocking pool") the local
has_initialized in credit_entropy_bits() won't get set anymore and
the corresponding if-clause became dead code. Remove it as well as the
has_initialized variable itself from credit_entropy_bits().

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index d20ba1b104ca..0580968fd28c 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -660,7 +660,7 @@ static void process_random_ready_list(void)
  */
 static void credit_entropy_bits(struct entropy_store *r, int nbits)
 {
-	int entropy_count, orig, has_initialized = 0;
+	int entropy_count, orig;
 	const int pool_size = r->poolinfo->poolfracbits;
 	int nfrac = nbits << ENTROPY_SHIFT;
 
@@ -717,11 +717,6 @@ static void credit_entropy_bits(struct entropy_store *r, int nbits)
 	if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
 		goto retry;
 
-	if (has_initialized) {
-		r->initialized = 1;
-		kill_fasync(&fasync, SIGIO, POLL_IN);
-	}
-
 	trace_credit_entropy_bits(r->name, nbits,
 				  entropy_count >> ENTROPY_SHIFT, _RET_IP_);
 
-- 
2.26.2

[RFC PATCH 02/41] random: remove dead code for nbits < 0 in credit_entropy_bits()

[Nicolai Stange]

The nbits argument to credit_entropy_bits() is never negative and
the branch handling it is dead code. Remove it.

The code for handling the regular nbits > 0 case used to live in the
corresponding else branch, but has now been lifted up to function scope.
Move the declaration of 'pnfrac' to the function prologue in order to
adhere to C99 rules. Likewise, move the declaration of 's' into the
body loop, the only scope it's referenced from.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 69 ++++++++++++++++++++-----------------------
 1 file changed, 32 insertions(+), 37 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 0580968fd28c..c4b7bdbd460e 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -654,7 +654,7 @@ static void process_random_ready_list(void)
 }
 
 /*
- * Credit (or debit) the entropy store with n bits of entropy.
+ * Credit the entropy store with n bits of entropy.
  * Use credit_entropy_bits_safe() if the value comes from userspace
  * or otherwise should be checked for extreme values.
  */
@@ -663,50 +663,45 @@ static void credit_entropy_bits(struct entropy_store *r, int nbits)
 	int entropy_count, orig;
 	const int pool_size = r->poolinfo->poolfracbits;
 	int nfrac = nbits << ENTROPY_SHIFT;
+	int pnfrac;
 
 	if (!nbits)
 		return;
 
 retry:
 	entropy_count = orig = READ_ONCE(r->entropy_count);
-	if (nfrac < 0) {
-		/* Debit */
-		entropy_count += nfrac;
-	} else {
-		/*
-		 * Credit: we have to account for the possibility of
-		 * overwriting already present entropy.	 Even in the
-		 * ideal case of pure Shannon entropy, new contributions
-		 * approach the full value asymptotically:
-		 *
-		 * entropy <- entropy + (pool_size - entropy) *
-		 *	(1 - exp(-add_entropy/pool_size))
-		 *
-		 * For add_entropy <= pool_size/2 then
-		 * (1 - exp(-add_entropy/pool_size)) >=
-		 *    (add_entropy/pool_size)*0.7869...
-		 * so we can approximate the exponential with
-		 * 3/4*add_entropy/pool_size and still be on the
-		 * safe side by adding at most pool_size/2 at a time.
-		 *
-		 * The use of pool_size-2 in the while statement is to
-		 * prevent rounding artifacts from making the loop
-		 * arbitrarily long; this limits the loop to log2(pool_size)*2
-		 * turns no matter how large nbits is.
-		 */
-		int pnfrac = nfrac;
-		const int s = r->poolinfo->poolbitshift + ENTROPY_SHIFT + 2;
+	/*
+	 * Credit: we have to account for the possibility of
+	 * overwriting already present entropy.	 Even in the
+	 * ideal case of pure Shannon entropy, new contributions
+	 * approach the full value asymptotically:
+	 *
+	 * entropy <- entropy + (pool_size - entropy) *
+	 *	(1 - exp(-add_entropy/pool_size))
+	 *
+	 * For add_entropy <= pool_size/2 then
+	 * (1 - exp(-add_entropy/pool_size)) >=
+	 *    (add_entropy/pool_size)*0.7869...
+	 * so we can approximate the exponential with
+	 * 3/4*add_entropy/pool_size and still be on the
+	 * safe side by adding at most pool_size/2 at a time.
+	 *
+	 * The use of pool_size-2 in the while statement is to
+	 * prevent rounding artifacts from making the loop
+	 * arbitrarily long; this limits the loop to log2(pool_size)*2
+	 * turns no matter how large nbits is.
+	 */
+	pnfrac = nfrac;
+	do {
 		/* The +2 corresponds to the /4 in the denominator */
+		const int s = r->poolinfo->poolbitshift + ENTROPY_SHIFT + 2;
+		unsigned int anfrac = min(pnfrac, pool_size/2);
+		unsigned int add =
+			((pool_size - entropy_count)*anfrac*3) >> s;
 
-		do {
-			unsigned int anfrac = min(pnfrac, pool_size/2);
-			unsigned int add =
-				((pool_size - entropy_count)*anfrac*3) >> s;
-
-			entropy_count += add;
-			pnfrac -= anfrac;
-		} while (unlikely(entropy_count < pool_size-2 && pnfrac));
-	}
+		entropy_count += add;
+		pnfrac -= anfrac;
+	} while (unlikely(entropy_count < pool_size-2 && pnfrac));
 
 	if (WARN_ON(entropy_count < 0)) {
 		pr_warn("negative entropy/overflow: pool %s count %d\n",
-- 
2.26.2

[RFC PATCH 03/41] random: prune dead assignment to entropy_bits in credit_entropy_bits()

[Nicolai Stange]

Since commit 90ea1c6436d2 ("random: remove the blocking pool"), the
last assignment to 'entropy_bits' is dead. Remove it.

While at it, move the declaration of 'entropy_bits' one scope down and make
it const: it's not used anywhere outside and never updated after
initialization.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index c4b7bdbd460e..14c39608cc17 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -716,13 +716,12 @@ static void credit_entropy_bits(struct entropy_store *r, int nbits)
 				  entropy_count >> ENTROPY_SHIFT, _RET_IP_);
 
 	if (r == &input_pool) {
-		int entropy_bits = entropy_count >> ENTROPY_SHIFT;
-
 		if (crng_init < 2) {
+			const int entropy_bits = entropy_count >> ENTROPY_SHIFT;
+
 			if (entropy_bits < 128)
 				return;
 			crng_reseed(&primary_crng, r);
-			entropy_bits = ENTROPY_BITS(r);
 		}
 	}
 }
-- 
2.26.2

[RFC PATCH 04/41] random: drop 'reserved' parameter from extract_entropy()

[Nicolai Stange]

Since commit 43d8a72cd985 ("random: remove variable limit") all call
sites of extract_entropy() pass in zero for the 'reserved' argument
and the corresponding code in account() is effectively dead.

Remove it and the drop the now unused 'reserved' argument from
extract_entropy() as well as from account() called therefrom.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 17 ++++++-----------
 1 file changed, 6 insertions(+), 11 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 14c39608cc17..35e381be20fe 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -506,7 +506,7 @@ struct entropy_store {
 };
 
 static ssize_t extract_entropy(struct entropy_store *r, void *buf,
-			       size_t nbytes, int min, int rsvd);
+			       size_t nbytes, int min);
 static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
 				size_t nbytes, int fips);
 
@@ -944,7 +944,7 @@ static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
 	} buf;
 
 	if (r) {
-		num = extract_entropy(r, &buf, 32, 16, 0);
+		num = extract_entropy(r, &buf, 32, 16);
 		if (num == 0)
 			return;
 	} else {
@@ -1330,8 +1330,7 @@ EXPORT_SYMBOL_GPL(add_disk_randomness);
  * This function decides how many bytes to actually take from the
  * given pool, and also debits the entropy count accordingly.
  */
-static size_t account(struct entropy_store *r, size_t nbytes, int min,
-		      int reserved)
+static size_t account(struct entropy_store *r, size_t nbytes, int min)
 {
 	int entropy_count, orig, have_bytes;
 	size_t ibytes, nfrac;
@@ -1345,8 +1344,6 @@ static size_t account(struct entropy_store *r, size_t nbytes, int min,
 	/* never pull more than available */
 	have_bytes = entropy_count >> (ENTROPY_SHIFT + 3);
 
-	if ((have_bytes -= reserved) < 0)
-		have_bytes = 0;
 	ibytes = min_t(size_t, ibytes, have_bytes);
 	if (ibytes < min)
 		ibytes = 0;
@@ -1469,12 +1466,10 @@ static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
  * returns it in a buffer.
  *
  * The min parameter specifies the minimum amount we can pull before
- * failing to avoid races that defeat catastrophic reseeding while the
- * reserved parameter indicates how much entropy we must leave in the
- * pool after each pull to avoid starving other readers.
+ * failing to avoid races that defeat catastrophic reseeding.
  */
 static ssize_t extract_entropy(struct entropy_store *r, void *buf,
-				 size_t nbytes, int min, int reserved)
+				 size_t nbytes, int min)
 {
 	__u8 tmp[EXTRACT_SIZE];
 	unsigned long flags;
@@ -1495,7 +1490,7 @@ static ssize_t extract_entropy(struct entropy_store *r, void *buf,
 	}
 
 	trace_extract_entropy(r->name, nbytes, ENTROPY_BITS(r), _RET_IP_);
-	nbytes = account(r, nbytes, min, reserved);
+	nbytes = account(r, nbytes, min);
 
 	return _extract_entropy(r, buf, nbytes, fips_enabled);
 }
-- 
2.26.2

[RFC PATCH 05/41] random: don't reset entropy to zero on overflow

[Nicolai Stange]

credit_entropy_bits() adds one or more positive values to the signed
entropy_count and checks if the result is negative afterwards. Note that
because the initial value of entropy_count is positive, a negative result
can happen only on overflow.

However, if the final entropy_count is found to have overflown, a WARN()
is emitted and the entropy_store's entropy count reset to zero. Even
though this case should never happen, it is better to retain previously
available entropy as this will facilitate a future change factoring out
that approximation of the exponential.

Make credit_entropy_bits() tp reset entropy_count to the original value
rather than zero on overflow.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 35e381be20fe..6adac462aa0d 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -706,7 +706,7 @@ static void credit_entropy_bits(struct entropy_store *r, int nbits)
 	if (WARN_ON(entropy_count < 0)) {
 		pr_warn("negative entropy/overflow: pool %s count %d\n",
 			r->name, entropy_count);
-		entropy_count = 0;
+		entropy_count = orig;
 	} else if (entropy_count > pool_size)
 		entropy_count = pool_size;
 	if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
-- 
2.26.2

[RFC PATCH 06/41] random: factor the exponential approximation in credit_entropy_bits() out

[Nicolai Stange]

In the course of calculating the actual amount of new entropy to credit,
credit_entropy_bits() applies a linear approximation to
exp(-nbits/pool_size)) (neglecting scaling factors in the exponent for
the sake of simplicity).

In order to limit approximation errors for large nbits, nbits is divided
into chunks of maximum value pool_size/2 each and said approximation is
applied to these individually in a loop. That loop has a theoretic upper
bound of 2*log2(pool_size), which, with the given pool_size of 128 * 32
bits, equals 24.

However, in practice nbits hardly ever exceeds values as a large as
pool_size/2 == 2048, especially not when called from interrupt context,
i.e. from add_interrupt_randomness() and alike. Thus, imposing a limit of
one single iteration in these contexts would yield a good guarantee with
respect to runtime while not losing any entropy.

In preparation to enabling that, move the approximation code in
credit_entropy_bits() into a separate function, pool_entropy_delta().
Based on the initial pool entropy count and the number of new entropy bits
to credit, it calculates and returns a (positive) delta to add to the
former. In case the 'fast' parameter is set to true, the calculation
will be terminated after the first iteration, effectively capping the input
nbits to one half of the pool size.

There is no functional change; callers with 'fast' set to true will be
introduced in a future patch.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 53 ++++++++++++++++++++++++++++++-------------
 1 file changed, 37 insertions(+), 16 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 6adac462aa0d..15dd22d74029 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -366,7 +366,7 @@
  * denominated in units of 1/8th bits.
  *
  * 2*(ENTROPY_SHIFT + poolbitshift) must <= 31, or the multiply in
- * credit_entropy_bits() needs to be 64 bits wide.
+ * pool_entropy_delta() needs to be 64 bits wide.
  */
 #define ENTROPY_SHIFT 3
 #define ENTROPY_BITS(r) ((r)->entropy_count >> ENTROPY_SHIFT)
@@ -654,22 +654,24 @@ static void process_random_ready_list(void)
 }
 
 /*
- * Credit the entropy store with n bits of entropy.
- * Use credit_entropy_bits_safe() if the value comes from userspace
- * or otherwise should be checked for extreme values.
+ * Based on the pool's current entropy fill level, specified as
+ * base_entropy_count, and the number of new entropy bits to add,
+ * return the amount of new entropy to credit. If the 'fast'
+ * parameter is set to true, the calculation will be guaranteed to
+ * terminate quickly, but this comes at the expense of capping
+ * nbits to one half of the pool size.
  */
-static void credit_entropy_bits(struct entropy_store *r, int nbits)
+static unsigned int pool_entropy_delta(struct entropy_store *r,
+				       int base_entropy_count,
+				       int nbits, bool fast)
 {
-	int entropy_count, orig;
 	const int pool_size = r->poolinfo->poolfracbits;
+	int entropy_count = base_entropy_count;
 	int nfrac = nbits << ENTROPY_SHIFT;
-	int pnfrac;
 
 	if (!nbits)
-		return;
+		return 0;
 
-retry:
-	entropy_count = orig = READ_ONCE(r->entropy_count);
 	/*
 	 * Credit: we have to account for the possibility of
 	 * overwriting already present entropy.	 Even in the
@@ -691,24 +693,43 @@ static void credit_entropy_bits(struct entropy_store *r, int nbits)
 	 * arbitrarily long; this limits the loop to log2(pool_size)*2
 	 * turns no matter how large nbits is.
 	 */
-	pnfrac = nfrac;
 	do {
 		/* The +2 corresponds to the /4 in the denominator */
 		const int s = r->poolinfo->poolbitshift + ENTROPY_SHIFT + 2;
-		unsigned int anfrac = min(pnfrac, pool_size/2);
+		unsigned int anfrac = min(nfrac, pool_size/2);
 		unsigned int add =
 			((pool_size - entropy_count)*anfrac*3) >> s;
 
 		entropy_count += add;
-		pnfrac -= anfrac;
-	} while (unlikely(entropy_count < pool_size-2 && pnfrac));
+		nfrac -= anfrac;
+	} while (unlikely(!fast && entropy_count < pool_size-2 && nfrac));
 
 	if (WARN_ON(entropy_count < 0)) {
 		pr_warn("negative entropy/overflow: pool %s count %d\n",
 			r->name, entropy_count);
-		entropy_count = orig;
-	} else if (entropy_count > pool_size)
+		entropy_count = base_entropy_count;
+	} else if (entropy_count > pool_size) {
 		entropy_count = pool_size;
+	}
+
+	return entropy_count - base_entropy_count;
+}
+
+/*
+ * Credit the entropy store with n bits of entropy.
+ * Use credit_entropy_bits_safe() if the value comes from userspace
+ * or otherwise should be checked for extreme values.
+ */
+static void credit_entropy_bits(struct entropy_store *r, int nbits)
+{
+	int entropy_count, orig;
+
+	if (!nbits)
+		return;
+
+retry:
+	orig = READ_ONCE(r->entropy_count);
+	entropy_count = orig + pool_entropy_delta(r, orig, nbits, false);
 	if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
 		goto retry;
 
-- 
2.26.2

[RFC PATCH 07/41] random: let pool_entropy_delta() take nbits in units of 2^-ENTROPY_SHIFT

[Nicolai Stange]

Currently pool_entropy_delta() expects its nbits argument to be given in
units of integral bits. Using fractional bits for processing intermediate
entropy counts consistently throughout the code will facilitate upcoming
changes to the entropy accounting logic in add_interrupt_randomness().
Replace pool_entropy_delta()'s nbits argument with nfrac, which used to be
a local variable and is expected to be given in units of 2^-ENTROPY_SHIFT.
Adapt the single caller, credit_entropy_bits(), accordingly.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 15dd22d74029..08caa7a691a5 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -655,21 +655,20 @@ static void process_random_ready_list(void)
 
 /*
  * Based on the pool's current entropy fill level, specified as
- * base_entropy_count, and the number of new entropy bits to add,
- * return the amount of new entropy to credit. If the 'fast'
- * parameter is set to true, the calculation will be guaranteed to
- * terminate quickly, but this comes at the expense of capping
- * nbits to one half of the pool size.
+ * base_entropy_count, and the number of new entropy bits in units of
+ * 2^-ENTROPY_SHIFT to add, return the amount of new entropy to
+ * credit. If the 'fast' parameter is set to true, the calculation
+ * will be guaranteed to terminate quickly, but this comes at the
+ * expense of capping nbits to one half of the pool size.
  */
 static unsigned int pool_entropy_delta(struct entropy_store *r,
 				       int base_entropy_count,
-				       int nbits, bool fast)
+				       int nfrac, bool fast)
 {
 	const int pool_size = r->poolinfo->poolfracbits;
 	int entropy_count = base_entropy_count;
-	int nfrac = nbits << ENTROPY_SHIFT;
 
-	if (!nbits)
+	if (!nfrac)
 		return 0;
 
 	/*
@@ -729,7 +728,9 @@ static void credit_entropy_bits(struct entropy_store *r, int nbits)
 
 retry:
 	orig = READ_ONCE(r->entropy_count);
-	entropy_count = orig + pool_entropy_delta(r, orig, nbits, false);
+	entropy_count = orig + pool_entropy_delta(r, orig,
+						  nbits << ENTROPY_SHIFT,
+						  false);
 	if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
 		goto retry;
 
-- 
2.26.2

[RFC PATCH 08/41] random: introduce __credit_entropy_bits_fast() for hot paths

[Nicolai Stange]

When transferring entropy from the fast_pool into the global input_pool
from add_interrupt_randomness(), there are at least two atomic operations
involved: one when taking the input_pool's spinlock for the actual mixing
and another one in the cmpxchg loop in credit_entropy_bits() for
updating the pool's ->entropy_count. Because cmpxchg is potentially costly,
it would be nice if it could be avoided.

As said, the input_pool's spinlock is taken anyway, and I see no reason
why its scope should not be extended to protect ->entropy_count as well.
Performance considerations set aside, this will also facilitate future
changes introducing additional fields to input_pool which will also have to
get updated atomically from the consumer/producer sides.

The actual move to extend the spinlock's scope to cover ->entropy_count
will be the subject of a future patch. Prepare for that by putting
a limit on the work to be done with the lock being held.

In order to avoid releasing and regrabbing from hot producer paths, they'll
keep the lock when executing those calculations in pool_entropy_delta().
The loop found in the latter has a theoretical upper bound of
2 * log2(pool_size) == 24 iterations. However, as all entropy increments
awarded from the interrupt path are less than pool_size/2 in magnitude,
it is safe to enforce a guaranteed limit of one on the iteration count
by setting pool_entropy_delta()'s 'fast' parameter.

Introduce __credit_entropy_bits_fast() doing exactly that. Currently
it resembles the behaviour from credit_entropy_bits() except that
- pool_entropy_delta() gets called with 'fast' set to true and
- that __credit_entropy_bits_fast() returns a bool indicating whether
  the caller should reseed the primary_crng.

Note that unlike it's the case with credit_entropy_bits(), the reseeding
won't be possible from within __credit_entropy_bits_fast() anymore once it
actually gets invoked with the pool lock being held in the future.

There is no functional change.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 49 ++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 46 insertions(+), 3 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 08caa7a691a5..d9e4dd27d45d 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -714,6 +714,39 @@ static unsigned int pool_entropy_delta(struct entropy_store *r,
 	return entropy_count - base_entropy_count;
 }
 
+/*
+ * Credit the entropy store with n bits of entropy.
+ * To be used from hot paths when it is either known that nbits is
+ * smaller than one half of the pool size or losing anything beyond that
+ * doesn't matter.
+ */
+static bool __credit_entropy_bits_fast(struct entropy_store *r, int nbits)
+{
+	int entropy_count, orig;
+
+	if (!nbits)
+		return false;
+
+retry:
+	orig = READ_ONCE(r->entropy_count);
+	entropy_count = orig + pool_entropy_delta(r, orig,
+						  nbits << ENTROPY_SHIFT,
+						  true);
+	if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
+		goto retry;
+
+	trace_credit_entropy_bits(r->name, nbits,
+				  entropy_count >> ENTROPY_SHIFT, _RET_IP_);
+
+	if (unlikely(r == &input_pool && crng_init < 2)) {
+		const int entropy_bits = entropy_count >> ENTROPY_SHIFT;
+
+		return (entropy_bits >= 128);
+	}
+
+	return false;
+}
+
 /*
  * Credit the entropy store with n bits of entropy.
  * Use credit_entropy_bits_safe() if the value comes from userspace
@@ -1169,6 +1202,7 @@ static void add_timer_randomness(struct timer_rand_state *state, unsigned num)
 		unsigned num;
 	} sample;
 	long delta, delta2, delta3;
+	bool reseed;
 
 	sample.jiffies = jiffies;
 	sample.cycles = random_get_entropy();
@@ -1206,7 +1240,9 @@ static void add_timer_randomness(struct timer_rand_state *state, unsigned num)
 	 * Round down by 1 bit on general principles,
 	 * and limit entropy estimate to 12 bits.
 	 */
-	credit_entropy_bits(r, min_t(int, fls(delta>>1), 11));
+	reseed = __credit_entropy_bits_fast(r, min_t(int, fls(delta>>1), 11));
+	if (reseed)
+		crng_reseed(&primary_crng, r);
 }
 
 void add_input_randomness(unsigned int type, unsigned int code,
@@ -1274,6 +1310,7 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	__u64			ip;
 	unsigned long		seed;
 	int			credit = 0;
+	bool			reseed;
 
 	if (cycles == 0)
 		cycles = get_reg(fast_pool, regs);
@@ -1326,7 +1363,9 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	fast_pool->count = 0;
 
 	/* award one bit for the contents of the fast pool */
-	credit_entropy_bits(r, credit + 1);
+	reseed = __credit_entropy_bits_fast(r, credit + 1);
+	if (reseed)
+		crng_reseed(&primary_crng, r);
 }
 EXPORT_SYMBOL_GPL(add_interrupt_randomness);
 
@@ -1599,7 +1638,11 @@ EXPORT_SYMBOL(get_random_bytes);
  */
 static void entropy_timer(struct timer_list *t)
 {
-	credit_entropy_bits(&input_pool, 1);
+	bool reseed;
+
+	reseed = __credit_entropy_bits_fast(&input_pool, 1);
+	if (reseed)
+		crng_reseed(&primary_crng, &input_pool);
 }
 
 /*
-- 
2.26.2

[RFC PATCH 09/41] random: protect ->entropy_count with the pool spinlock

[Nicolai Stange]

Currently, all updates to ->entropy_count are synchronized by means of
cmpxchg-retry loops found in credit_entropy_bits(),
__credit_entropy_bits_fast() and account() respectively.

However, all but one __credit_entropy_bits_fast() call sites grap the pool
->lock already and it would be nice if the potentially costly cmpxchg could
be avoided in these performance critical paths. In addition to that, future
patches will introduce new fields to struct entropy_store which will
required some kinf of synchronization with ->entropy_count updates from
said producer paths as well.

Protect ->entropy_count with the pool ->lock.

- Make callers of __credit_entropy_bits_fast() invoke it with the
  pool ->lock held. Extend existing critical sections where possible.
  Drop the cmpxchg-reply loop in __credit_entropy_bits_fast() in favor of
  a plain assignment.
- Retain the retry loop in credit_entropy_bits(): the potentially
  expensive pool_entropy_delta() should not be called under the lock in
  order to not unnecessarily block contenders. In order to continue to
  synchronize with  __credit_entropy_bits_fast() and account(), the
  cmpxchg gets replaced by a plain comparison + store with the ->lock being
  held.
- Make account() grab the ->lock and drop the cmpxchg-retry loop in favor
  of a plain assignent.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 44 +++++++++++++++++++++++++++++--------------
 1 file changed, 30 insertions(+), 14 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index d9e4dd27d45d..9f87332b158f 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -718,7 +718,7 @@ static unsigned int pool_entropy_delta(struct entropy_store *r,
  * Credit the entropy store with n bits of entropy.
  * To be used from hot paths when it is either known that nbits is
  * smaller than one half of the pool size or losing anything beyond that
- * doesn't matter.
+ * doesn't matter. Must be called with r->lock being held.
  */
 static bool __credit_entropy_bits_fast(struct entropy_store *r, int nbits)
 {
@@ -727,13 +727,11 @@ static bool __credit_entropy_bits_fast(struct entropy_store *r, int nbits)
 	if (!nbits)
 		return false;
 
-retry:
-	orig = READ_ONCE(r->entropy_count);
+	orig = r->entropy_count;
 	entropy_count = orig + pool_entropy_delta(r, orig,
 						  nbits << ENTROPY_SHIFT,
 						  true);
-	if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
-		goto retry;
+	WRITE_ONCE(r->entropy_count, entropy_count);
 
 	trace_credit_entropy_bits(r->name, nbits,
 				  entropy_count >> ENTROPY_SHIFT, _RET_IP_);
@@ -755,17 +753,28 @@ static bool __credit_entropy_bits_fast(struct entropy_store *r, int nbits)
 static void credit_entropy_bits(struct entropy_store *r, int nbits)
 {
 	int entropy_count, orig;
+	unsigned long flags;
 
 	if (!nbits)
 		return;
 
 retry:
+	/*
+	 * Don't run the potentially expensive pool_entropy_delta()
+	 * calculations under the spinlock. Instead retry until
+	 * ->entropy_count becomes stable.
+	 */
 	orig = READ_ONCE(r->entropy_count);
 	entropy_count = orig + pool_entropy_delta(r, orig,
 						  nbits << ENTROPY_SHIFT,
 						  false);
-	if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
+	spin_lock_irqsave(&r->lock, flags);
+	if (r->entropy_count != orig) {
+		spin_unlock_irqrestore(&r->lock, flags);
 		goto retry;
+	}
+	WRITE_ONCE(r->entropy_count, entropy_count);
+	spin_unlock_irqrestore(&r->lock, flags);
 
 	trace_credit_entropy_bits(r->name, nbits,
 				  entropy_count >> ENTROPY_SHIFT, _RET_IP_);
@@ -1203,12 +1212,11 @@ static void add_timer_randomness(struct timer_rand_state *state, unsigned num)
 	} sample;
 	long delta, delta2, delta3;
 	bool reseed;
+	unsigned long flags;
 
 	sample.jiffies = jiffies;
 	sample.cycles = random_get_entropy();
 	sample.num = num;
-	r = &input_pool;
-	mix_pool_bytes(r, &sample, sizeof(sample));
 
 	/*
 	 * Calculate number of bits of randomness we probably added.
@@ -1235,12 +1243,16 @@ static void add_timer_randomness(struct timer_rand_state *state, unsigned num)
 	if (delta > delta3)
 		delta = delta3;
 
+	r = &input_pool;
+	spin_lock_irqsave(&r->lock, flags);
+	__mix_pool_bytes(r, &sample, sizeof(sample));
 	/*
 	 * delta is now minimum absolute delta.
 	 * Round down by 1 bit on general principles,
 	 * and limit entropy estimate to 12 bits.
 	 */
 	reseed = __credit_entropy_bits_fast(r, min_t(int, fls(delta>>1), 11));
+	spin_unlock_irqrestore(&r->lock, flags);
 	if (reseed)
 		crng_reseed(&primary_crng, r);
 }
@@ -1358,12 +1370,12 @@ void add_interrupt_randomness(int irq, int irq_flags)
 		__mix_pool_bytes(r, &seed, sizeof(seed));
 		credit = 1;
 	}
-	spin_unlock(&r->lock);
 
 	fast_pool->count = 0;
 
 	/* award one bit for the contents of the fast pool */
 	reseed = __credit_entropy_bits_fast(r, credit + 1);
+	spin_unlock(&r->lock);
 	if (reseed)
 		crng_reseed(&primary_crng, r);
 }
@@ -1393,14 +1405,15 @@ EXPORT_SYMBOL_GPL(add_disk_randomness);
  */
 static size_t account(struct entropy_store *r, size_t nbytes, int min)
 {
-	int entropy_count, orig, have_bytes;
+	int entropy_count, have_bytes;
 	size_t ibytes, nfrac;
+	unsigned long flags;
 
 	BUG_ON(r->entropy_count > r->poolinfo->poolfracbits);
 
+	spin_lock_irqsave(&r->lock, flags);
 	/* Can we pull enough? */
-retry:
-	entropy_count = orig = READ_ONCE(r->entropy_count);
+	entropy_count = r->entropy_count;
 	ibytes = nbytes;
 	/* never pull more than available */
 	have_bytes = entropy_count >> (ENTROPY_SHIFT + 3);
@@ -1420,8 +1433,8 @@ static size_t account(struct entropy_store *r, size_t nbytes, int min)
 	else
 		entropy_count = 0;
 
-	if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
-		goto retry;
+	WRITE_ONCE(r->entropy_count, entropy_count);
+	spin_unlock_irqrestore(&r->lock, flags);
 
 	trace_debit_entropy(r->name, 8 * ibytes);
 	if (ibytes && ENTROPY_BITS(r) < random_write_wakeup_bits) {
@@ -1639,8 +1652,11 @@ EXPORT_SYMBOL(get_random_bytes);
 static void entropy_timer(struct timer_list *t)
 {
 	bool reseed;
+	unsigned long flags;
 
+	spin_lock_irqsave(&input_pool.lock, flags);
 	reseed = __credit_entropy_bits_fast(&input_pool, 1);
+	spin_unlock_irqrestore(&input_pool.lock, flags);
 	if (reseed)
 		crng_reseed(&primary_crng, &input_pool);
 }
-- 
2.26.2

[RFC PATCH 10/41] random: implement support for delayed entropy dispatching

[Nicolai Stange]

Consider the following scenario:

  Producer                                        Consumer
  --------                                        --------
  mix_pool_bytes()
                                                  account()
                                                    ->entropy_count -= n
                                                  extract_buf()
  credit_entropy_bits()
    ->entropy_count += pool_entropy_delta()

The amount of entropy to credit as calculated by pool_entropy_delta()
depends on the current pool fill level: the higher the current
->entropy_count, the less the amount of new entropy credited. In the
situation above, a too small value of ->entropy_count would have been
observed and thus, too much entropy attributed to the new batch.

I do recognize the fact that this is currently more of a theoretical
concern. However, future patches will implement some statistical "health
tests" to be run on raw samples like e.g. cycle counts obtained and mixed
into the fast_pools in add_interrupt_randomness(). These tests must have
processed more events than can fit into the fast_pools (~64) before the
outcome is known. Thus, add_interrupt_randomness() will have to dump its
fast_pool into the global input_pool a couple of times before the tests
have completed and hence before the (accumulated) entropy credit may be
released to input_pool's ->entropy_count. It follows that the final entropy
credit attribution can be delayed for arbitrarily long to after the
corresponding mix_pool_bytes() operation.

The simplest solution would be to maintain a sequence counter which gets
incremented from account(). The producer side would take a snapshot before
mix_pool_bytes() and only eventually credit any entropy if it hasn't
changed in the meanwhile. However, that would mean that a lot of precious
entropy would be discarded, especially at boot time: as soon as the first
CPU seeds the primary_crng(), a large part of the entropy accumulated
through add_interrupt_randomness() on all other CPUs would be lost.

So follow a watermark based approach instead. That is, provide the producer
side with an ->entropy_count watermark which is guaranteed to not be less
than the value of ->entropy_count at any point in time from before to after
the mix_pool_bytes() operation(s). Note that the order in which concurrent
producers credit entropy doesn't matter, because
  e1 = e0 + pool_entropy_delta(e0, n1)
  e2 = e1 + pool_entropy_delta(e1, n2)
is equivalent (modulo approximation artifacts) to
  e2 = e0 + pool_entropy_delta(e0, n1 + n2).
Thus, taking the larger of said watermark and the latest ->entropy_count
value for the pool fill level when calculating pool_entropy_delta() will
guarantee that the result won't exceed the true value.

Introduce the new __queue_entropy() and __dequeue_entropy() functions
intended to be used for delimiting one or more successive mix_pool_bytes()
invocations for which the pool watermark tracking is needed. Both take a
pointer to the target pool as well as to an instance of the new
struct queued_entropy. For reasons explained below, __queue_entropy() also
receives the amount of entropy transferred in the subsequent
mix_pool_bytes() operation as an argument and accumulates that at the given
struct queued_entropy instance. __queue_entropy() may be called any number
of times on the same struct queued_entropy instance until a matching
__dequeue_entropy() gets eventually invoked. The latter will return the
total number of (fractional) entropy bits accumulated at queued_entropy as
well as an appropriate pool watermark. Both are intended to be used for
that pool_entropy_delta() calculation when subsequently dispatching the
accumulated entropy to the pool.

Producers are not actually expected to call __dequeue_entropy() directly.
Instead, provide the new dispatch_queued_entropy() and
__dispatch_queued_entropy_fast() helpers. These will eventually supersede
credit_entropy_bits() respectively __credit_entropy_bits_fast(). Both take
a queued_entropy instance, run __dequeue_entropy() on it, carry out the
required pool_entropy_delta() calculations and add the result to the
target pool's ->entropy_count. Conversion of the individual entropy
producers to the new API will be the subject of future patches for the sake
of better reviewability. For now, merely reimplement credit_entropy_bits()
and __credit_entropy_bits_fast() on top of it in order to avoid excessive
code duplication.

Obviously, it's the consumer side's job to maintain the pool watermark:
whenever ->entropy_count decreases, the watermark needs updating. Maintain
the pool entropy watermark in the form of a delta to be added to the
current ->entropy_count to obtain the actual value. To this end, introduce
a new field ->entropy_watermark_delta to struct entropy_store.

Rename the existing account(), which gets called right before the
corresponding extract_buf()s in extract_entropy(), to account_begin(). Make
it add the allocated entropy count, i.e. the amount by which the pool's
->entropy_count has been reduced, to ->entropy_watermark_delta.

If possible, this watermark increment should be undone after the subsequent
extract_buf()s have completed, because otherwise the watermark would grow
unboundedly beyond the pool size over time. Note that this would render
producers unable to dispatch any new non-zero entropy to ->entropy_count.
Introduce the new account_complete() for handling the
->entropy_watermark_delta decrements and call it from extract_entropy()
right after the extract_buf()s following the preceding account_begin()
have finished.

Obviously it's safe to decrement the watermark again in case nobody cares
at all -- that is, if there currently isn't any entropy queued at the
producer side. Introduce a new field ->queued_entropy_batches to struct
entropy_store for keeping track of that. Make __queue_entropy() increment
it upon the first request to queue a non-zero amount of entropy at a given
struct queued_entropy instance. Likewise, make __dequeue_entropy()
decrement it again iff a non-zero amount of entropy has been queued.
Finally, make account_complete() undo the ->entropy_watermark_delta
increment from the preceding account_begin() in case
->queued_entropy_batches is zero.

Note that if ->queued_entropy_batches is found to be non-zero in
account_complete(), ->entropy_watermark_delta is left untouched, i.e. the
increment from the preceding account_begin() is "leaked". It follows
that the watermark can still grow beyond any bound over time. However, if
at the time account_complete() runs there is no entropy queued at the
producer side *and* there is no other, concurrent extraction pending an
upcoming __queue_entropy() could possibly interfere with, the watermark may
even get reset to zero and thus, any leakages left from former invocations
cleaned up. Introduce a new field ->pending_extractions to
struct entropy_store for keeping track of the number of pending entropy
extractions. Make account_begin() increment it and make account_complete()
decrement it again. Make account_complete() reset ->entropy_watermark_delta
in case ->queued_entropy_batches and ->entropy_watermark_delta are both
zero.

Once the initially mentioned health tests have been implemented and
enabled, it will not be unlikely that there's always at least one CPU
having some entropy queued at any point in time and thus, that
->queued_entropy_batches will never be found to equal zero in
account_complete(). As a last resort, enforce upper bounds on the magnitude
as well as on the lifetime of the pool watermark and reset it if any has
been exceeded. All entropy currently queued up on the producer side needs
to be invalidated in this case. Introduce a new field
->entropy_watermark_seq to struct entropy_store for maintaing a sequence
count needed to implement entropy invalidations. Make __queue_entropy()
take a snapshot at the first invocation and make it revalidate the
snapshot when accumulating additional entropy in subsequent invocations.
Make the final __dequeue_entropy() validate the snapshot and return zero
for the amount of dequeued entropy on failure. Make account_complete()
increment the sequence count when resetting the pool watermark even though
->queued_entropy_batches is non-zero.

Note that this sequence count based invalidation scheme does not allow
for watermark resets when there are multiple concurrent extractions
pending: a future __queue_entropy() could potentially interfere with any
of the other extractions and there is no way to invalidate it "in advance".
However, this doesn't matter because there are hardly any concurrent
entropy extractions after boot and even if there were: some
account_complete() would always come last.

What remains to be clarified is under which exact circumstances
account_complete() would resort to resetting the pool watermark and
invalidating all currently queued entropy. The limit on the watermark
magnitude, ->entropy_watermark_delta to be more specific, has been set to
one eighth of the pool size == 1/8 * 4096 bits == 512 bits. This has been
chosen as a compromise between allowing for up to two 256 bit
extractions/reseeds without invalidating queued entropy and not reducing
the efficiency of new entropy contributions too much. Assuming a watermark
value of 512 bits over the current ->entropy_count, the entropy credits as
calculated by pool_entropy_delta() for new contributions are 87.5%, 75%
and 50% respectively for pool fill levels of 0%, 50% and 75% of what they
would have been with a ->entropy_watermark_delta of zero. In order to avoid
a situation where a smallish ->entropy_watermark_delta which accumulated
during boot time, but never manages to increase beyond the reset threshold,
is kept forever, also impose a lifetime limit. The choice of
2 * CRNG_RESEED_INTERVAL for the maximum watermark lifetime follows the
same line of reasoning as for the chosen magnitude limit.

In order to enable this watermark lifetime management, add yet another new
field ->entropy_watermark_leak_time to struct entropy_store. Make
account_begin() set it to the current jiffies upon the first increment of
->entropy_watermark_delta from zero. Make account_complete() reset
->entropy_watermark_delta and invalidate all queued entropy as
described above whenever ->pending_extractions is zero and either
->entropy_watermark_leak_time is older than two times CRNG_RESEED_INTERVAL
or if ->entropy_watermark_delta exceeds one fourth of the pool size.

As entropy producers haven't been converted to the new __queue_entropy() +
dispatch_queued_entropy()/__dispatch_queued_entropy_fast() API yet, the net
effect of this patch is to "fix" a scenario similar to the one initially
described for those producers that call __mix_pool_bytes() and
__credit_entropy_bits_fast() without dropping the pool's ->lock inbetween,
i.e. for add_interrupt_randomness() and add_timer_randomness(). Namely, if
said sequence happens to get serialized inbetween the account_begin()
(formerly account()) and the last extract_buf() from a concurrent
extraction, the pool's entropy watermark will now be taken into account
when calculating the amount of new entropy to credit in
__credit_entropy_bits_fast(), because the latter has been reimplemented on
top of the new API.

Other than that, it's noteworthy that the pool entropy watermark might
exceed unexpectedly high levels at boot time, namely if multiple producers
happen to trigger the initial seeding of the primary_crng and the
subsequent entropy extractions complete when entropy has been queued up
somewhere else, e.g. in try_to_generate_entropy(). As detailed above, high
values of the pool watermark will reduce the efficiency when dispatching
new entropy attributions, but note that
- There are mechanisms in place to limit the effect in magnitude and
  time.
- The watermark can never exceed the total amount of entropy collected
  so far. So entropy collection at boot time would have to be terribly
  efficient in order for this to matter.
- As seeding the primary_crng is a prerequisite for the described scenario,
  a temporarily reduced entropy collection efficiency isn't really
  concerning: getting towards a seeded primary_crng is all that matters at
  this point.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 315 +++++++++++++++++++++++++++++++++++++++---
 1 file changed, 292 insertions(+), 23 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 9f87332b158f..b91d1fc08ac5 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -499,7 +499,13 @@ struct entropy_store {
 	spinlock_t lock;
 	unsigned short add_ptr;
 	unsigned short input_rotate;
+
 	int entropy_count;
+	unsigned int entropy_watermark_delta;
+	unsigned int entropy_watermark_seq;
+	unsigned int queued_entropy_batches;
+	unsigned int pending_extractions;
+	unsigned long entropy_watermark_leak_time;
 	unsigned int initialized:1;
 	unsigned int last_data_init:1;
 	__u8 last_data[EXTRACT_SIZE];
@@ -671,6 +677,9 @@ static unsigned int pool_entropy_delta(struct entropy_store *r,
 	if (!nfrac)
 		return 0;
 
+	if (pool_size <= base_entropy_count)
+		return 0;
+
 	/*
 	 * Credit: we have to account for the possibility of
 	 * overwriting already present entropy.	 Even in the
@@ -714,26 +723,172 @@ static unsigned int pool_entropy_delta(struct entropy_store *r,
 	return entropy_count - base_entropy_count;
 }
 
+struct queued_entropy {
+	unsigned int pool_watermark_seq;
+	unsigned int queued_entropy_fracbits;
+};
+
 /*
- * Credit the entropy store with n bits of entropy.
- * To be used from hot paths when it is either known that nbits is
- * smaller than one half of the pool size or losing anything beyond that
- * doesn't matter. Must be called with r->lock being held.
+ * Queue a given amount of new entropy which is about to mixed into
+ * the entropy pool for later dispatch.
+ *
+ * __queue_entropy() may be called one or more time on the same struct
+ * queued_entropy instance in order to accumulate entropy for later
+ * dispatch. However, any such sequence of invocations must eventually
+ * be followed by exactly one call to either of __dequeue_entropy(),
+ * __dispatch_queued_entropy_fast() or dispatch_queued_entropy()
+ * when the actual pool mixing has completed.
+ * __queue_entropy() must be called with r->lock held.
+ *
+ * Entropy extraction is a two-step process:
+ * 1.) The allocated amount of entropy gets subtracted from ->entropy_count.
+ * 2.) The entropy is actually extracted from the pool by means of one or more
+ *     extract_buf() invocations.
+ * Likewise for the mixing side:
+ * 1.) The new entropy data gets mixed into the pool via __mix_pool_bytes() and
+ * 2.) the pool's ->entropy_count incremented by a certain amount afterwards.
+ * However, that amount of new entropy credited in the last step depends
+ * on the current pool fill level: the higher the current ->entropy_count,
+ * the less the amount of new entropy credited, c.f. pool_entropy_delta().
+ *
+ * This must be accounted for in a scenario involving concurrent producers
+ * and consumers like the following:
+ *   Producer                                     Consumer
+ *   --------                                     --------
+ *                                                ->entropy_count -= n
+ *   __mix_pool_bytes()
+ *   ->entropy_count += pool_entropy_delta()
+ *                                                extract_buf()
+ * Note how the pool_entropy_delta() would observe a too small pool
+ * fill level and thus, credits too much entropy to the new batch.
+ *
+ * The solution to work around this is to maintain a watermark, which
+ * is guaranteed to be >= than the pool's ->entropy_count value
+ * at any point in time from before __mix_pool_bytes() to after it.
+ *
+ * A call to __queue_entropy() enables watermark tracking from the
+ * producers side, the final __dequeue_entropy() disables it and
+ * returns the watermark. See also account_begin() and
+ * account_complete().
+ *
+ * Note there's no problem wuth multiple concurrent producers, because
+ *   e1 = e0 + pool_entropy_delta(e0, n1);
+ *   e2 = e1 + pool_entropy_delta(e1, n2);
+ * is equivalent (modulo approximation artifacts) to
+ *   e2 = e0 + pool_entropy_delta(e0, n1 + n2);
  */
-static bool __credit_entropy_bits_fast(struct entropy_store *r, int nbits)
+static void __queue_entropy(struct entropy_store *r, struct queued_entropy *q,
+			    unsigned int nfrac)
+{
+	if (!nfrac)
+		return;
+
+	if (!q->queued_entropy_fracbits) {
+		/*
+		 * First call with non-zero nbits, enable watermark
+		 * tracking.
+		 */
+		q->pool_watermark_seq = r->entropy_watermark_seq;
+		r->queued_entropy_batches++;
+	} else if (q->pool_watermark_seq != r->entropy_watermark_seq) {
+		/*
+		 * Previously queued entropy is doomed because
+		 * the ->pool_watermark_delta had been reset.
+		 * Don't add any more entropy on top of that.
+		 */
+		q->pool_watermark_seq = r->entropy_watermark_seq;
+		q->queued_entropy_fracbits = 0;
+	}
+
+	q->queued_entropy_fracbits += nfrac;
+}
+
+static void queue_entropy(struct entropy_store *r, struct queued_entropy *q,
+			  unsigned int nfrac)
 {
-	int entropy_count, orig;
+	unsigned long flags;
 
-	if (!nbits)
+	spin_lock_irqsave(&r->lock, flags);
+	__queue_entropy(r, q, nfrac);
+	spin_unlock_irqrestore(&r->lock, flags);
+}
+
+/*
+ * Dequeue previously queued entropy and return the pool entropy
+ * watermark to be used in pool_entropy_delta().
+ *
+ * Must only be called after a sequence of one or more matching
+ * __queue_entropy() invocations. Must be called with r->lock
+ * held.
+ *
+ * __dequeue_entropy() returns the number of queued bits and resets
+ * q. *pool_watermark receives the pool entropy watermark as tracked
+ * from the beginning of the first preceding __queue_entropy() call
+ * up to the __dequeue_entropy() invocation.
+ *
+ * The number of returned fractional bits is intended to get
+ * subsequently passed together with the larger of *pool_watermark and
+ * r->entropy_count to pool_entropy_delta().
+ * If r->lock is not dropped inbetween *pool_watermark and the load
+ * of r->entropy_count, the former is guaranteed to equal the maximum.
+ */
+static unsigned int __dequeue_entropy(struct entropy_store *r,
+				      struct queued_entropy *q,
+				      int *pool_watermark)
+{
+	unsigned int nfrac;
+
+	nfrac = q->queued_entropy_fracbits;
+	if (!nfrac)
+		return 0;
+
+	/* Disable watermark tracking. */
+	q->queued_entropy_fracbits = 0;
+	r->queued_entropy_batches--;
+
+	/*
+	 * The watermark has been invalidated in the meanwhile and
+	 * the queued entropy is lost.
+	 */
+	if (q->pool_watermark_seq != r->entropy_watermark_seq)
+		return 0;
+
+	*pool_watermark = r->entropy_count + r->entropy_watermark_delta;
+	if (*pool_watermark < 0)
+		return 0;
+
+	return nfrac;
+}
+
+/*
+ * Credit the pool with previously queued entropy.
+ *
+ * Must only be called after a sequence of one or more matching
+ * __queue_entropy() invocations. Must be called with r->lock
+ * held.
+ *
+ * To be used from hot paths when it is either known that the amount
+ * of queued entropy is smaller than one half of the pool size or
+ * losing anything beyond that doesn't matter.
+ *
+ * Returns true if the caller is supposed to seed the primary_crng.
+ */
+static bool __dispatch_queued_entropy_fast(struct entropy_store *r,
+					   struct queued_entropy *q)
+{
+	unsigned int nfrac;
+	int entropy_count, orig, pool_watermark;
+
+	nfrac = __dequeue_entropy(r, q, &pool_watermark);
+	if (!nfrac)
 		return false;
 
 	orig = r->entropy_count;
-	entropy_count = orig + pool_entropy_delta(r, orig,
-						  nbits << ENTROPY_SHIFT,
+	entropy_count = orig + pool_entropy_delta(r, pool_watermark, nfrac,
 						  true);
 	WRITE_ONCE(r->entropy_count, entropy_count);
 
-	trace_credit_entropy_bits(r->name, nbits,
+	trace_credit_entropy_bits(r->name, nfrac >> ENTROPY_SHIFT,
 				  entropy_count >> ENTROPY_SHIFT, _RET_IP_);
 
 	if (unlikely(r == &input_pool && crng_init < 2)) {
@@ -747,15 +902,35 @@ static bool __credit_entropy_bits_fast(struct entropy_store *r, int nbits)
 
 /*
  * Credit the entropy store with n bits of entropy.
- * Use credit_entropy_bits_safe() if the value comes from userspace
- * or otherwise should be checked for extreme values.
+ * To be used from hot paths when it is either known that nbits is
+ * smaller than one half of the pool size or losing anything beyond that
+ * doesn't matter. Must be called with r->lock being held.
  */
-static void credit_entropy_bits(struct entropy_store *r, int nbits)
+static bool __credit_entropy_bits_fast(struct entropy_store *r, int nbits)
+{
+	struct queued_entropy q = { 0 };
+
+	__queue_entropy(r, &q, nbits << ENTROPY_SHIFT);
+	return __dispatch_queued_entropy_fast(r, &q);
+}
+
+/*
+ * Credit the pool with previously queued entropy.
+ *
+ * Must only be called after a sequence of one or more matching
+ * __queue_entropy() invocations.
+ */
+static void dispatch_queued_entropy(struct entropy_store *r,
+				    struct queued_entropy *q)
 {
-	int entropy_count, orig;
+	unsigned int nfrac;
+	int entropy_count, orig, pool_watermark, base;
 	unsigned long flags;
 
-	if (!nbits)
+	spin_lock_irqsave(&r->lock, flags);
+	nfrac = __dequeue_entropy(r, q, &pool_watermark);
+	spin_unlock_irqrestore(&r->lock, flags);
+	if (!nfrac)
 		return;
 
 retry:
@@ -765,9 +940,8 @@ static void credit_entropy_bits(struct entropy_store *r, int nbits)
 	 * ->entropy_count becomes stable.
 	 */
 	orig = READ_ONCE(r->entropy_count);
-	entropy_count = orig + pool_entropy_delta(r, orig,
-						  nbits << ENTROPY_SHIFT,
-						  false);
+	base = max_t(int, pool_watermark, orig);
+	entropy_count = orig + pool_entropy_delta(r, base, nfrac, false);
 	spin_lock_irqsave(&r->lock, flags);
 	if (r->entropy_count != orig) {
 		spin_unlock_irqrestore(&r->lock, flags);
@@ -776,7 +950,7 @@ static void credit_entropy_bits(struct entropy_store *r, int nbits)
 	WRITE_ONCE(r->entropy_count, entropy_count);
 	spin_unlock_irqrestore(&r->lock, flags);
 
-	trace_credit_entropy_bits(r->name, nbits,
+	trace_credit_entropy_bits(r->name, nfrac >> ENTROPY_SHIFT,
 				  entropy_count >> ENTROPY_SHIFT, _RET_IP_);
 
 	if (r == &input_pool) {
@@ -790,6 +964,19 @@ static void credit_entropy_bits(struct entropy_store *r, int nbits)
 	}
 }
 
+/*
+ * Credit the entropy store with n bits of entropy.
+ * Use credit_entropy_bits_safe() if the value comes from userspace
+ * or otherwise should be checked for extreme values.
+ */
+static void credit_entropy_bits(struct entropy_store *r, int nbits)
+{
+	struct queued_entropy q = { 0 };
+
+	queue_entropy(r, &q, nbits << ENTROPY_SHIFT);
+	dispatch_queued_entropy(r, &q);
+}
+
 static int credit_entropy_bits_safe(struct entropy_store *r, int nbits)
 {
 	const int nbits_max = r->poolinfo->poolwords * 32;
@@ -1402,8 +1589,12 @@ EXPORT_SYMBOL_GPL(add_disk_randomness);
 /*
  * This function decides how many bytes to actually take from the
  * given pool, and also debits the entropy count accordingly.
+ *
+ * Increases the pool entropy watermark (c.f. __queue_entropy() and
+ * __dequeue_entropy()) and must be followed with a matching
+ * account_complete() in order to decrease it again, if possible.
  */
-static size_t account(struct entropy_store *r, size_t nbytes, int min)
+static size_t account_begin(struct entropy_store *r, size_t nbytes, int min)
 {
 	int entropy_count, have_bytes;
 	size_t ibytes, nfrac;
@@ -1419,6 +1610,7 @@ static size_t account(struct entropy_store *r, size_t nbytes, int min)
 	have_bytes = entropy_count >> (ENTROPY_SHIFT + 3);
 
 	ibytes = min_t(size_t, ibytes, have_bytes);
+
 	if (ibytes < min)
 		ibytes = 0;
 
@@ -1434,6 +1626,18 @@ static size_t account(struct entropy_store *r, size_t nbytes, int min)
 		entropy_count = 0;
 
 	WRITE_ONCE(r->entropy_count, entropy_count);
+
+	if (!r->entropy_watermark_delta) {
+		/*
+		 * This is not exact. In fact it is not known yet if
+		 * the watermark entropy added below will be actually
+		 * be leaked in account_complete(). But there can be
+		 * concurrent consumers and someone has to set this.
+		 */
+		r->entropy_watermark_leak_time = jiffies;
+	}
+	r->entropy_watermark_delta += nfrac;
+	r->pending_extractions++;
 	spin_unlock_irqrestore(&r->lock, flags);
 
 	trace_debit_entropy(r->name, 8 * ibytes);
@@ -1445,6 +1649,69 @@ static size_t account(struct entropy_store *r, size_t nbytes, int min)
 	return ibytes;
 }
 
+/*
+ * Undo the pool watermark increment from a preceding
+ * account_begin(), if possible.
+ */
+static void account_complete(struct entropy_store *r, size_t ibytes)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&r->lock, flags);
+	r->pending_extractions--;
+	if (!r->queued_entropy_batches) {
+		/*
+		 * There's currently no entropy queued at the producer
+		 * side and at the very least it is safe to undo the
+		 * watermark increment from the matching
+		 * account_begin().
+		 */
+		if (!r->pending_extractions) {
+			/*
+			 * No other extractions pending. It is even
+			 * safe to dismiss all watermark increments
+			 * which had to be leaked from previous,
+			 * unrelated account_complete() invocations
+			 * because there had been some entropy queued
+			 * at their time.
+			 */
+			r->entropy_watermark_delta = 0;
+		} else {
+			unsigned int nfrac;
+
+			nfrac = ibytes << (ENTROPY_SHIFT + 3);
+			r->entropy_watermark_delta -= nfrac;
+		}
+	} else if (!r->pending_extractions) {
+		/*
+		 * There is currently some entropy queued at the
+		 * producer side and there's no choice but to leave
+		 * the pool watermark untouched and thus, to "leak"
+		 * the increment from the matching account_begin().
+		 *
+		 * However, if it gets too wild, the watermark is
+		 * reset and all currently queued entropy invalidated.
+		 * We don't want to keep leaked watermark increments
+		 * forever and also keep them bounded by 1/8 of the
+		 * pool size in total in order to limit its damping
+		 * effect on new entropy in pool_entropy_delta().
+		 */
+		int leak_limit;
+		unsigned long leak_cleanup_time;
+
+		leak_limit = r->poolinfo->poolfracbits >> 3;
+		leak_cleanup_time = (r->entropy_watermark_leak_time +
+				     2 * CRNG_RESEED_INTERVAL);
+		if (r->entropy_watermark_delta > leak_limit ||
+		    time_after(jiffies, leak_cleanup_time)) {
+			r->entropy_watermark_delta = 0;
+			/* Invalidate all queued entropy. */
+			r->entropy_watermark_seq++;
+		}
+	}
+	spin_unlock_irqrestore(&r->lock, flags);
+}
+
 /*
  * This function does the actual extraction for extract_entropy and
  * extract_entropy_user.
@@ -1547,6 +1814,7 @@ static ssize_t extract_entropy(struct entropy_store *r, void *buf,
 {
 	__u8 tmp[EXTRACT_SIZE];
 	unsigned long flags;
+	ssize_t ret;
 
 	/* if last_data isn't primed, we need EXTRACT_SIZE extra bytes */
 	if (fips_enabled) {
@@ -1564,9 +1832,10 @@ static ssize_t extract_entropy(struct entropy_store *r, void *buf,
 	}
 
 	trace_extract_entropy(r->name, nbytes, ENTROPY_BITS(r), _RET_IP_);
-	nbytes = account(r, nbytes, min);
-
-	return _extract_entropy(r, buf, nbytes, fips_enabled);
+	nbytes = account_begin(r, nbytes, min);
+	ret = _extract_entropy(r, buf, nbytes, fips_enabled);
+	account_complete(r, nbytes);
+	return ret;
 }
 
 #define warn_unseeded_randomness(previous) \
-- 
2.26.2

[RFC PATCH 11/41] random: convert add_timer_randomness() to queued_entropy API

[Nicolai Stange]

In an effort to drop __credit_entropy_bits_fast() in favor of the new
__queue_entropy()/__dispatch_queued_entropy_fast() API, convert
add_timer_randomness() from the former to the latter.

There is no change in functionality at this point, because
__credit_entropy_bits_fast() has already been reimplemented on top of the
new API before.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index b91d1fc08ac5..e8c86abde901 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1400,6 +1400,7 @@ static void add_timer_randomness(struct timer_rand_state *state, unsigned num)
 	long delta, delta2, delta3;
 	bool reseed;
 	unsigned long flags;
+	struct queued_entropy q = { 0 };
 
 	sample.jiffies = jiffies;
 	sample.cycles = random_get_entropy();
@@ -1432,13 +1433,14 @@ static void add_timer_randomness(struct timer_rand_state *state, unsigned num)
 
 	r = &input_pool;
 	spin_lock_irqsave(&r->lock, flags);
-	__mix_pool_bytes(r, &sample, sizeof(sample));
 	/*
 	 * delta is now minimum absolute delta.
 	 * Round down by 1 bit on general principles,
 	 * and limit entropy estimate to 12 bits.
 	 */
-	reseed = __credit_entropy_bits_fast(r, min_t(int, fls(delta>>1), 11));
+	__queue_entropy(r, &q, min_t(int, fls(delta>>1), 11) << ENTROPY_SHIFT);
+	__mix_pool_bytes(r, &sample, sizeof(sample));
+	reseed = __dispatch_queued_entropy_fast(r, &q);
 	spin_unlock_irqrestore(&r->lock, flags);
 	if (reseed)
 		crng_reseed(&primary_crng, r);
-- 
2.26.2

[RFC PATCH 12/41] random: convert add_interrupt_randomness() to queued_entropy API

[Nicolai Stange]

In an effort to drop __credit_entropy_bits_fast() in favor of the new
__queue_entropy()/__dispatch_queued_entropy_fast() API, convert
add_interrupt_randomness() from the former to the latter.

There is no change in functionality at this point, because
__credit_entropy_bits_fast() has already been reimplemented on top of the
new API before.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index e8c86abde901..bd3774c6be4b 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1512,6 +1512,7 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	unsigned long		seed;
 	int			credit = 0;
 	bool			reseed;
+	struct queued_entropy	q = { 0 };
 
 	if (cycles == 0)
 		cycles = get_reg(fast_pool, regs);
@@ -1546,24 +1547,27 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	if (!spin_trylock(&r->lock))
 		return;
 
-	fast_pool->last = now;
-	__mix_pool_bytes(r, &fast_pool->pool, sizeof(fast_pool->pool));
-
 	/*
 	 * If we have architectural seed generator, produce a seed and
-	 * add it to the pool.  For the sake of paranoia don't let the
-	 * architectural seed generator dominate the input from the
-	 * interrupt noise.
+	 * add it to the pool further below. For the sake of paranoia
+	 * don't let the architectural seed generator dominate the
+	 * input from the interrupt noise.
 	 */
-	if (arch_get_random_seed_long(&seed)) {
-		__mix_pool_bytes(r, &seed, sizeof(seed));
-		credit = 1;
-	}
+	credit = !!arch_get_random_long(&seed);
 
+	fast_pool->last = now;
 	fast_pool->count = 0;
-
 	/* award one bit for the contents of the fast pool */
-	reseed = __credit_entropy_bits_fast(r, credit + 1);
+	__queue_entropy(r, &q, (credit + 1) << ENTROPY_SHIFT);
+	__mix_pool_bytes(r, &fast_pool->pool, sizeof(fast_pool->pool));
+	if (credit) {
+		/*
+		 * A seed has been obtained from
+		 * arch_get_random_seed_long() above, mix it in.
+		 */
+		__mix_pool_bytes(r, &seed, sizeof(seed));
+	}
+	reseed = __dispatch_queued_entropy_fast(r, &q);
 	spin_unlock(&r->lock);
 	if (reseed)
 		crng_reseed(&primary_crng, r);
-- 
2.26.2

[RFC PATCH 13/41] random: convert try_to_generate_entropy() to queued_entropy API

[Nicolai Stange]

In an effort to drop __credit_entropy_bits_fast() in favor of the new
__queue_entropy()/__dispatch_queued_entropy_fast() API, convert
try_to_generate_entropy() from the former to the latter.

Replace the call to __credit_entropy_bits_fast() from the timer callback,
entropy_timer(), by a queue_entropy() operation. Dispatch it from the loop
in try_to_generate_entropy() by invoking __dispatch_queued_entropy_fast()
after the timestamp has been mixed into the input_pool.

In order to provide the timer callback and try_to_generate_entropy() with
access to a common struct queued_entropy instance, move the currently
anonymous struct definition from the local 'stack' variable declaration in
try_to_generate_entropy() to file scope and assign it a name,
"struct try_to_generate_entropy_stack". Make entropy_timer() obtain a
pointer to the corresponding instance by means of container_of() on the
->timer member contained therein. Amend struct
try_to_generate_entropy_stack by a new member ->q of type struct
queued_entropy.

Note that the described scheme alters behaviour a bit: first of all, new
entropy credit now gets only dispatched to the pool after the actual mixing
has completed rather than in an unsynchronized manner directly from the
timer callback. As the mixing loop try_to_generate_entropy() is expected to
run at higher frequency than the timer, this is unlikely to make any
difference in practice.

Furthermore, the pool entropy watermark as tracked over the period from
queuing the entropy in the timer callback and to its subsequent dispatch
from try_to_generate_entropy() is now taken into account when calculating
the actual credit at dispatch. In consequence, the amount of new entropy
dispatched to the pool will potentially be lowered if said period happens
to overlap with the pool extraction from an initial crng_reseed() on the
primary_crng. However, as getting the primary_crng seeded is the whole
point of the try_to_generate_entropy() exercise, this won't matter.

Note that instead of calling queue_entropy() from the timer callback,
an alternative would have been to maintain an invocation counter and queue
that up from try_to_generate_entropy() right before the mix operation.
This would have reduced the described effect of the pool's entropy
watermark and in fact matched the intended queue_entropy() API usage
better. However, in this particular case of try_to_generate_entropy(),
jitter is desired and invoking queue_entropy() with its buffer locking etc.
from the timer callback could potentially contribute to that.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 42 +++++++++++++++++++++++++++++-------------
 1 file changed, 29 insertions(+), 13 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index bd3774c6be4b..dfbe49fdbcf1 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1911,6 +1911,12 @@ void get_random_bytes(void *buf, int nbytes)
 EXPORT_SYMBOL(get_random_bytes);
 
 
+struct try_to_generate_entropy_stack {
+	unsigned long now;
+	struct timer_list timer;
+	struct queued_entropy q;
+} stack;
+
 /*
  * Each time the timer fires, we expect that we got an unpredictable
  * jump in the cycle counter. Even if the timer is running on another
@@ -1926,14 +1932,10 @@ EXPORT_SYMBOL(get_random_bytes);
  */
 static void entropy_timer(struct timer_list *t)
 {
-	bool reseed;
-	unsigned long flags;
+	struct try_to_generate_entropy_stack *stack;
 
-	spin_lock_irqsave(&input_pool.lock, flags);
-	reseed = __credit_entropy_bits_fast(&input_pool, 1);
-	spin_unlock_irqrestore(&input_pool.lock, flags);
-	if (reseed)
-		crng_reseed(&primary_crng, &input_pool);
+	stack = container_of(t, struct try_to_generate_entropy_stack, timer);
+	queue_entropy(&input_pool, &stack->q, 1 << ENTROPY_SHIFT);
 }
 
 /*
@@ -1942,10 +1944,9 @@ static void entropy_timer(struct timer_list *t)
  */
 static void try_to_generate_entropy(void)
 {
-	struct {
-		unsigned long now;
-		struct timer_list timer;
-	} stack;
+	struct try_to_generate_entropy_stack stack = { 0 };
+	unsigned long flags;
+	bool reseed;
 
 	stack.now = random_get_entropy();
 
@@ -1957,14 +1958,29 @@ static void try_to_generate_entropy(void)
 	while (!crng_ready()) {
 		if (!timer_pending(&stack.timer))
 			mod_timer(&stack.timer, jiffies+1);
-		mix_pool_bytes(&input_pool, &stack.now, sizeof(stack.now));
+		spin_lock_irqsave(&input_pool.lock, flags);
+		__mix_pool_bytes(&input_pool, &stack.now, sizeof(stack.now));
+		reseed = __dispatch_queued_entropy_fast(&input_pool, &stack.q);
+		spin_unlock_irqrestore(&input_pool.lock, flags);
+
+		if (reseed)
+			crng_reseed(&primary_crng, &input_pool);
+
 		schedule();
 		stack.now = random_get_entropy();
 	}
 
 	del_timer_sync(&stack.timer);
 	destroy_timer_on_stack(&stack.timer);
-	mix_pool_bytes(&input_pool, &stack.now, sizeof(stack.now));
+	spin_lock_irqsave(&input_pool.lock, flags);
+	__mix_pool_bytes(&input_pool, &stack.now, sizeof(stack.now));
+	/*
+	 * Must be called here once more in order to complete a
+	 * previously unmatched queue_entropy() from entropy_timer(),
+	 * if any.
+	 */
+	__dispatch_queued_entropy_fast(&input_pool, &stack.q);
+	spin_unlock_irqrestore(&input_pool.lock, flags);
 }
 
 /*
-- 
2.26.2

[RFC PATCH 14/41] random: drop __credit_entropy_bits_fast()

[Nicolai Stange]

All former call sites of __credit_entropy_bits_fast() have been converted
to the new __dispatch_queued_entropy_fast() API. Drop the now unused
__credit_entropy_bits_fast().

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 14 --------------
 1 file changed, 14 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index dfbe49fdbcf1..60ce185d7b2d 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -900,20 +900,6 @@ static bool __dispatch_queued_entropy_fast(struct entropy_store *r,
 	return false;
 }
 
-/*
- * Credit the entropy store with n bits of entropy.
- * To be used from hot paths when it is either known that nbits is
- * smaller than one half of the pool size or losing anything beyond that
- * doesn't matter. Must be called with r->lock being held.
- */
-static bool __credit_entropy_bits_fast(struct entropy_store *r, int nbits)
-{
-	struct queued_entropy q = { 0 };
-
-	__queue_entropy(r, &q, nbits << ENTROPY_SHIFT);
-	return __dispatch_queued_entropy_fast(r, &q);
-}
-
 /*
  * Credit the pool with previously queued entropy.
  *
-- 
2.26.2

[RFC PATCH 15/41] random: convert add_hwgenerator_randomness() to queued_entropy API

[Nicolai Stange]

In an effort to drop credit_entropy_bits() in favor of the new
queue_entropy()/dispatch_queued_entropy() API, convert
add_hwgenerator_randomness() from the former to the latter.

As a side effect, the pool entropy watermark as tracked over the duration
of the mix_pool_bytes() operation is now taken correctly taken into account
when calulating the amount of new entropy to dispatch to the pool based on
the latter's fill level.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 60ce185d7b2d..78e65367ea86 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -2640,6 +2640,7 @@ void add_hwgenerator_randomness(const char *buffer, size_t count,
 				size_t entropy)
 {
 	struct entropy_store *poolp = &input_pool;
+	struct queued_entropy q = { 0 };
 
 	if (unlikely(crng_init == 0)) {
 		crng_fast_load(buffer, count);
@@ -2652,8 +2653,9 @@ void add_hwgenerator_randomness(const char *buffer, size_t count,
 	 */
 	wait_event_interruptible(random_write_wait, kthread_should_stop() ||
 			ENTROPY_BITS(&input_pool) <= random_write_wakeup_bits);
+	queue_entropy(poolp, &q, entropy << ENTROPY_SHIFT);
 	mix_pool_bytes(poolp, buffer, count);
-	credit_entropy_bits(poolp, entropy);
+	dispatch_queued_entropy(poolp, &q);
 }
 EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
 
-- 
2.26.2

[RFC PATCH 16/41] random: convert random_ioctl() to queued_entropy API

[Nicolai Stange]

In an effort to drop credit_entropy_bits_safe() in favor of the new
queue_entropy()/dispatch_queued_entropy() API, convert random_ioctl() from
the former to the latter.

Implement two helpers:
- queue_entropy_bits_safe(), which checks the entropy passed from userspace
  for extreme values in analogy to what credit_entropy_bits_safe() did
- discard_queue_entropy(), which is invoked from random_ioctly() to discard
  the entropy queued prior to the write_pool() call in case the latter
  fails.

Use them to convert the two call sites of credit_entropy_bits_safe()
in random_ioctl() to the new API.

As a side effect, the pool entropy watermark as tracked over the duration
of the write_pool() operation is now taken correctly taken into account
when calulating the amount of new entropy to dispatch to the pool based on
the latter's fill level.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 57 ++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 53 insertions(+), 4 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 78e65367ea86..03eadefabbca 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -737,7 +737,9 @@ struct queued_entropy {
  * dispatch. However, any such sequence of invocations must eventually
  * be followed by exactly one call to either of __dequeue_entropy(),
  * __dispatch_queued_entropy_fast() or dispatch_queued_entropy()
- * when the actual pool mixing has completed.
+ * when the actual pool mixing has completed. Alternatively,
+ * discard_queued_entropy() may be called in case the mixing has
+ * failed.
  * __queue_entropy() must be called with r->lock held.
  *
  * Entropy extraction is a two-step process:
@@ -813,6 +815,26 @@ static void queue_entropy(struct entropy_store *r, struct queued_entropy *q,
 	spin_unlock_irqrestore(&r->lock, flags);
 }
 
+/*
+ * Queue entropy which comes from userspace and might take extreme
+ * values.
+ */
+static int queue_entropy_bits_safe(struct entropy_store *r,
+				   struct queued_entropy *q,
+				   int nbits)
+{
+	const int nbits_max = r->poolinfo->poolwords * 32;
+
+	if (nbits < 0)
+		return -EINVAL;
+
+	/* Cap the value to avoid overflows */
+	nbits = min(nbits,  nbits_max);
+
+	queue_entropy(r, q, nbits << ENTROPY_SHIFT);
+	return 0;
+}
+
 /*
  * Dequeue previously queued entropy and return the pool entropy
  * watermark to be used in pool_entropy_delta().
@@ -950,6 +972,22 @@ static void dispatch_queued_entropy(struct entropy_store *r,
 	}
 }
 
+/*
+ * Discard queued entropy. May be called when e.g. a write_pool()
+ * operation failed and the corresponding previously queued entropy
+ * should not get dispatched to the pool.
+ */
+static void discard_queued_entropy(struct entropy_store *r,
+				   struct queued_entropy *q)
+{
+	unsigned long flags;
+	int pool_watermark;
+
+	spin_lock_irqsave(&r->lock, flags);
+	__dequeue_entropy(r, q, &pool_watermark);
+	spin_unlock_irqrestore(&r->lock, flags);
+}
+
 /*
  * Credit the entropy store with n bits of entropy.
  * Use credit_entropy_bits_safe() if the value comes from userspace
@@ -2272,6 +2310,7 @@ static long random_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
 	int size, ent_count;
 	int __user *p = (int __user *)arg;
 	int retval;
+	struct queued_entropy q = { 0 };
 
 	switch (cmd) {
 	case RNDGETENTCNT:
@@ -2285,7 +2324,11 @@ static long random_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
 			return -EPERM;
 		if (get_user(ent_count, p))
 			return -EFAULT;
-		return credit_entropy_bits_safe(&input_pool, ent_count);
+		retval = queue_entropy_bits_safe(&input_pool, &q, ent_count);
+		if (retval < 0)
+			return retval;
+		dispatch_queued_entropy(&input_pool, &q);
+		return 0;
 	case RNDADDENTROPY:
 		if (!capable(CAP_SYS_ADMIN))
 			return -EPERM;
@@ -2295,11 +2338,17 @@ static long random_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
 			return -EINVAL;
 		if (get_user(size, p++))
 			return -EFAULT;
+		retval = queue_entropy_bits_safe(&input_pool, &q, ent_count);
+		if (retval < 0)
+			return retval;
 		retval = write_pool(&input_pool, (const char __user *)p,
 				    size);
-		if (retval < 0)
+		if (retval < 0) {
+			discard_queued_entropy(&input_pool, &q);
 			return retval;
-		return credit_entropy_bits_safe(&input_pool, ent_count);
+		}
+		discard_queued_entropy(&input_pool, &q);
+		return 0;
 	case RNDZAPENTCNT:
 	case RNDCLEARPOOL:
 		/*
-- 
2.26.2

[RFC PATCH 17/41] random: drop credit_entropy_bits() and credit_entropy_bits_safe()

[Nicolai Stange]

All former call sites of credit_entropy_bits() and
credit_entropy_bits_safe() respectively have been converted to the new
dispatch_queued_entropy() API. Drop the now unused functions.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 29 +----------------------------
 1 file changed, 1 insertion(+), 28 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 03eadefabbca..a49805d0d23c 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -533,7 +533,7 @@ static __u32 const twist_table[8] = {
 /*
  * This function adds bytes into the entropy "pool".  It does not
  * update the entropy estimate.  The caller should call
- * credit_entropy_bits if this is appropriate.
+ * queue_entropy()+dispatch_queued_entropy() if this is appropriate.
  *
  * The pool is stirred with a primitive polynomial of the appropriate
  * degree, and then twisted.  We twist by three bits at a time because
@@ -988,33 +988,6 @@ static void discard_queued_entropy(struct entropy_store *r,
 	spin_unlock_irqrestore(&r->lock, flags);
 }
 
-/*
- * Credit the entropy store with n bits of entropy.
- * Use credit_entropy_bits_safe() if the value comes from userspace
- * or otherwise should be checked for extreme values.
- */
-static void credit_entropy_bits(struct entropy_store *r, int nbits)
-{
-	struct queued_entropy q = { 0 };
-
-	queue_entropy(r, &q, nbits << ENTROPY_SHIFT);
-	dispatch_queued_entropy(r, &q);
-}
-
-static int credit_entropy_bits_safe(struct entropy_store *r, int nbits)
-{
-	const int nbits_max = r->poolinfo->poolwords * 32;
-
-	if (nbits < 0)
-		return -EINVAL;
-
-	/* Cap the value to avoid overflows */
-	nbits = min(nbits,  nbits_max);
-
-	credit_entropy_bits(r, nbits);
-	return 0;
-}
-
 /*********************************************************************
  *
  * CRNG using CHACHA20
-- 
2.26.2

[RFC PATCH 18/41] random: move arch_get_random_seed() calls in crng_reseed() into own loop

[Nicolai Stange]

x86's RDSEED/RDRAND insns have reportedly been slowed down significantly
due to the ucode update required to mitigate against the "Special Register
Buffer Data Sampling" vulnerability (CVE-2020-0543) and should not get
invoked from the interrupt path anymore.

In preparation of getting rid of that arch_get_random_long() call currently
found in add_interrupt_randomness(), move those arch_get_random_long()
calls in crng_reseed() into a separate loop and outside of the crng->lock.

There is no functional change.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index a49805d0d23c..1945249597e0 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1200,14 +1200,18 @@ static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
 		_crng_backtrack_protect(&primary_crng, buf.block,
 					CHACHA_KEY_SIZE);
 	}
-	spin_lock_irqsave(&crng->lock, flags);
+
 	for (i = 0; i < 8; i++) {
 		unsigned long	rv;
 		if (!arch_get_random_seed_long(&rv) &&
 		    !arch_get_random_long(&rv))
 			rv = random_get_entropy();
-		crng->state[i+4] ^= buf.key[i] ^ rv;
+		buf.key[i] ^= rv;
 	}
+
+	spin_lock_irqsave(&crng->lock, flags);
+	for (i = 0; i < 8; i++)
+		crng->state[i+4] ^= buf.key[i];
 	memzero_explicit(&buf, sizeof(buf));
 	crng->init_time = jiffies;
 	spin_unlock_irqrestore(&crng->lock, flags);
-- 
2.26.2

[RFC PATCH 19/41] random: reintroduce arch_has_random() + arch_has_random_seed()

[Nicolai Stange]

A future patch will introduce support for making up for a certain amount
of lacking entropy in crng_reseed() by means of arch_get_random_long()
or arch_get_random_seed_long() respectively.

However, before even the tiniest bit of precious entropy is withdrawn from
the input_pool, it should be checked if whether the current arch even
has support for these.

Reintroduce arch_has_random() + arch_has_random_seed() and implement
them for arm64, powerpc, s390 and x86 as appropriate (yeah, I know this
should go in separate commits, but this is part of a RFC series).

Note that this more or less reverts commits
  647f50d5d9d9 ("linux/random.h: Remove arch_has_random,
                 arch_has_random_seed")
  cbac004995a0 ("powerpc: Remove arch_has_random, arch_has_random_seed")
  5e054c820f59 ("s390: Remove arch_has_random, arch_has_random_seed")
  5f2ed7f5b99b ("x86: Remove arch_has_random, arch_has_random_seed")

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 arch/arm64/include/asm/archrandom.h   | 25 ++++++++++++++++++-------
 arch/powerpc/include/asm/archrandom.h | 12 +++++++++++-
 arch/s390/include/asm/archrandom.h    | 14 ++++++++++++--
 arch/x86/include/asm/archrandom.h     | 18 ++++++++++++++----
 include/linux/random.h                |  8 ++++++++
 5 files changed, 63 insertions(+), 14 deletions(-)

diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h
index 44209f6146aa..055d18713db7 100644
--- a/arch/arm64/include/asm/archrandom.h
+++ b/arch/arm64/include/asm/archrandom.h
@@ -26,17 +26,13 @@ static inline bool __arm64_rndr(unsigned long *v)
 	return ok;
 }
 
-static inline bool __must_check arch_get_random_long(unsigned long *v)
-{
-	return false;
-}
 
-static inline bool __must_check arch_get_random_int(unsigned int *v)
+static inline bool arch_has_random(void)
 {
 	return false;
 }
 
-static inline bool __must_check arch_get_random_seed_long(unsigned long *v)
+static inline bool arch_has_random_seed(void)
 {
 	/*
 	 * Only support the generic interface after we have detected
@@ -44,7 +40,22 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v)
 	 * cpufeature code and with potential scheduling between CPUs
 	 * with and without the feature.
 	 */
-	if (!cpus_have_const_cap(ARM64_HAS_RNG))
+	return cpus_have_const_cap(ARM64_HAS_RNG);
+}
+
+static inline bool __must_check arch_get_random_long(unsigned long *v)
+{
+	return false;
+}
+
+static inline bool __must_check arch_get_random_int(unsigned int *v)
+{
+	return false;
+}
+
+static inline bool __must_check arch_get_random_seed_long(unsigned long *v)
+{
+	if (!arch_has_random_seed())
 		return false;
 
 	return __arm64_rndr(v);
diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h
index 9a53e29680f4..47c2d74e7244 100644
--- a/arch/powerpc/include/asm/archrandom.h
+++ b/arch/powerpc/include/asm/archrandom.h
@@ -6,6 +6,16 @@
 
 #include <asm/machdep.h>
 
+static inline bool arch_has_random(void)
+{
+	return false;
+}
+
+static inline bool arch_has_random_seed(void)
+{
+	return ppc_md.get_random_seed;
+}
+
 static inline bool __must_check arch_get_random_long(unsigned long *v)
 {
 	return false;
@@ -18,7 +28,7 @@ static inline bool __must_check arch_get_random_int(unsigned int *v)
 
 static inline bool __must_check arch_get_random_seed_long(unsigned long *v)
 {
-	if (ppc_md.get_random_seed)
+	if (arch_has_random_seed())
 		return ppc_md.get_random_seed(v);
 
 	return false;
diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h
index de61ce562052..18973845634c 100644
--- a/arch/s390/include/asm/archrandom.h
+++ b/arch/s390/include/asm/archrandom.h
@@ -21,6 +21,16 @@ extern atomic64_t s390_arch_random_counter;
 
 bool s390_arch_random_generate(u8 *buf, unsigned int nbytes);
 
+static inline bool arch_has_random(void)
+{
+	return false;
+}
+
+static inline bool arch_has_random_seed(void)
+{
+	return static_branch_likely(&s390_arch_random_available);
+}
+
 static inline bool __must_check arch_get_random_long(unsigned long *v)
 {
 	return false;
@@ -33,7 +43,7 @@ static inline bool __must_check arch_get_random_int(unsigned int *v)
 
 static inline bool __must_check arch_get_random_seed_long(unsigned long *v)
 {
-	if (static_branch_likely(&s390_arch_random_available)) {
+	if (arch_has_random_seed()) {
 		return s390_arch_random_generate((u8 *)v, sizeof(*v));
 	}
 	return false;
@@ -41,7 +51,7 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v)
 
 static inline bool __must_check arch_get_random_seed_int(unsigned int *v)
 {
-	if (static_branch_likely(&s390_arch_random_available)) {
+	if (arch_has_random_seed()) {
 		return s390_arch_random_generate((u8 *)v, sizeof(*v));
 	}
 	return false;
diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h
index ebc248e49549..030f46c9e310 100644
--- a/arch/x86/include/asm/archrandom.h
+++ b/arch/x86/include/asm/archrandom.h
@@ -70,24 +70,34 @@ static inline bool __must_check rdseed_int(unsigned int *v)
  */
 #ifdef CONFIG_ARCH_RANDOM
 
+static inline bool arch_has_random(void)
+{
+	return static_cpu_has(X86_FEATURE_RDRAND);
+}
+
+static inline bool arch_has_random_seed(void)
+{
+	return static_cpu_has(X86_FEATURE_RDSEED);
+}
+
 static inline bool __must_check arch_get_random_long(unsigned long *v)
 {
-	return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_long(v) : false;
+	return arch_has_random() ? rdrand_long(v) : false;
 }
 
 static inline bool __must_check arch_get_random_int(unsigned int *v)
 {
-	return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_int(v) : false;
+	return arch_has_random() ? rdrand_int(v) : false;
 }
 
 static inline bool __must_check arch_get_random_seed_long(unsigned long *v)
 {
-	return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_long(v) : false;
+	return arch_has_random_seed() ? rdseed_long(v) : false;
 }
 
 static inline bool __must_check arch_get_random_seed_int(unsigned int *v)
 {
-	return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_int(v) : false;
+	return arch_has_random_seed() ? rdseed_int(v) : false;
 }
 
 extern void x86_init_rdrand(struct cpuinfo_x86 *c);
diff --git a/include/linux/random.h b/include/linux/random.h
index f45b8be3e3c4..d4653422a0c7 100644
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -120,6 +120,14 @@ unsigned long randomize_page(unsigned long start, unsigned long range);
 #ifdef CONFIG_ARCH_RANDOM
 # include <asm/archrandom.h>
 #else
+static inline bool arch_has_random(void)
+{
+	return false;
+}
+static inline bool arch_has_random_seed(void)
+{
+	return false;
+}
 static inline bool __must_check arch_get_random_long(unsigned long *v)
 {
 	return false;
-- 
2.26.2

[RFC PATCH 20/41] random: provide min_crng_reseed_pool_entropy()

[Nicolai Stange]

Currently, the current minimum entropy required from the input_pool for
reseeding the primary_crng() is 16 bytes == 128 bits. A future patch will
introduce support for obtaining up to a certain fraction thereof from the
architecture's RNG, if available.

This will effectively lower the minimum input_pool ->entropy_count required
for a successful reseed of the primary_crng.

As this value is used at a couple of places, namely crng_reseed() itself
as well as dispatch_queued_entropy() and __dispatch_queued_entropy_fast(),
introduce min_crng_reseed_pool_entropy() to ensure consistency among
these.

min_crng_reseed_pool_entropy() returns the minimum amount of entropy in
bytes required from the input_pool for a successful reseed of the
primary_crng. Currently it's hardcoded to 16.

Use it in place of the hardcoded constants in crng_reseed(),
dispatch_queued_entropy() and __dispatch_queued_entropy_fast().

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 1945249597e0..424de1565927 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -516,6 +516,8 @@ static ssize_t extract_entropy(struct entropy_store *r, void *buf,
 static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
 				size_t nbytes, int fips);
 
+static int min_crng_reseed_pool_entropy(void);
+
 static void crng_reseed(struct crng_state *crng, struct entropy_store *r);
 static __u32 input_pool_data[INPUT_POOL_WORDS] __latent_entropy;
 
@@ -916,7 +918,7 @@ static bool __dispatch_queued_entropy_fast(struct entropy_store *r,
 	if (unlikely(r == &input_pool && crng_init < 2)) {
 		const int entropy_bits = entropy_count >> ENTROPY_SHIFT;
 
-		return (entropy_bits >= 128);
+		return (entropy_bits >= min_crng_reseed_pool_entropy() * 8);
 	}
 
 	return false;
@@ -965,7 +967,7 @@ static void dispatch_queued_entropy(struct entropy_store *r,
 		if (crng_init < 2) {
 			const int entropy_bits = entropy_count >> ENTROPY_SHIFT;
 
-			if (entropy_bits < 128)
+			if (entropy_bits < min_crng_reseed_pool_entropy() * 8)
 				return;
 			crng_reseed(&primary_crng, r);
 		}
@@ -1182,6 +1184,15 @@ static int crng_slow_load(const char *cp, size_t len)
 	return 1;
 }
 
+/*
+ * Minimum amount of entropy in bytes required from the input_pool for
+ * a successful reseed of the primary_crng.
+ */
+static int min_crng_reseed_pool_entropy(void)
+{
+	return 16;
+}
+
 static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
 {
 	unsigned long	flags;
@@ -1192,7 +1203,8 @@ static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
 	} buf;
 
 	if (r) {
-		num = extract_entropy(r, &buf, 32, 16);
+		num = extract_entropy(r, &buf, 32,
+				      min_crng_reseed_pool_entropy());
 		if (num == 0)
 			return;
 	} else {
-- 
2.26.2

[RFC PATCH 21/41] random: don't invoke arch_get_random_long() from add_interrupt_randomness()

[Nicolai Stange]

x86's RDSEED/RDRAND insns have reportedly been slowed down significantly
on certain CPU families due to the ucode update required to mitigate
against the "Special Register Buffer Data Sampling" vulnerability
(CVE-2020-0543) and should not get invoked from the interrupt path anymore.

Currently, add_interrupt_randomness() obtains an arch_get_random_long()
sample for each bit of entropy awarded to the "interrupt source",
mixes it into the input_pool and awards that sample another bit of entropy.
This lock step between the interrupt source and arch_get_random_long()
ensures that the latter cannot dominate the former.

There are some more entropy sources all mixing into input_pool with a
non-zero entropy attribution:
- try_to_generate_entropy() at boot time
- add_input_randomness() and add_disk_randomness()
- add_hwgenerator_randomness().
I don't see what's so special about the interrupt randomness source that
entropy awarded to the architectural RNG should be limited to its output
rate only rather than to the joint rate from all these entropy sources as
a whole.

Follow this approach. Don't mix arch_get_random_long() entropy from
add_interrupt_randomness() into the input_pool. Instead, make crng_reseed()
invoke the architectural RNG to make up for any lack of entropy up to one
half of the minimum seed size. That is, if the input_pool contains less
than 128 bits of entropy, the architectural RNG will be invoked and
attributed an entropy value equal to the difference, but never more than
64 bits. Note that
- the architectural RNG won't be able to dominate the other randomness
  sources taken together in this scheme and
- in case the input_pool contains more entropy than required for the
  minimum seed level, it won't be attributed any entropy at all.
That is, the architectural RNG is effectively turned into an emergency
reserve in a sense.

A potentially adverse effect of this change is that entropy might get
depleted at a higher rate than before from the interrupt source, namely if
the input_pool contains more than half of the minimum seed size of entropy
at reseed. However, the entropy sources feeding into input_pool are assumed
to provide entropy at a steady rate when averaged over the time scale of a
reseed interval, which is several minutes in length. Thus, as the
primary_crng reseeds are the only consumers of input_pool entropy nowadays,
the input_pool's fill level can be assumed to be relatively constant at the
time of reseeds and an equilibrium between the rates at which the
input_pool receives and releases entropy will be reached.

OTOH, remember that the rate at which the pool entropy increases is
exponentially damped as the pool fills up. In case the interrupt source is
a major contributor to the pool, not having to account anymore for the
architectural RNG's noise formerly mixed in in lockstep will leave
considerably more pool capacity to the interrupt noise, which is a welcomed
side effect.

So, make min_crng_reseed_pool_entropy() return only half of the minimum
seed size required in case an architectural RNG will likely be able to
provide the other half, as indicated by
arch_has_random() || arch_has_random_seed(). This will effectively
- make dispatch_queued_entropy() to attempt an intitial seed of the
  primary_crng as soon as the amount of entropy available from the
  input_pool has first exceeded that threshold and also
- makes crng_reseed() to lower the minimum amount of entropy to be
  extracted from the input_pool by means of extract_entropy() to one half
  of the minimum seed size.

Introduce a new boolean variable "arch_randomness_required" to
crng_reseed() for tracking whether or not the seed must be amended by
additional output from the architectural RNG. Initialize it to false,
make crng_reseed() set it in case its extract_entropy() invocation could
obtain only less than the minimum seed size from input_pool.

crng_reseed() already attempts to xor output from the architectural RNG
over the full length of the crng state, i.e. over the full length of the
latter's 256 bit ChaCha20 key. Currently, failure in doing so is not
considered fatal. Make it so if arch_randomness_required has been set.

Note that assuming one bit of entropy per bit obtained from the
architectural RNG, it would actually suffice to successfully obtain
(16 - num + sizeof(u32) - 1) / sizeof(u32) u32's from
arch_get_random_long()/arch_get_random_seed_long(), where 16 is the
minimum seed size in bytes and num is the number of bytes which have
been previuously obtained from the input_pool. However, this assumption
might be overly optimistic and the total number of arch_get_random_long()
invocations per 64 bits of entropy attributed to it has already been
lowered from >= 64 to eight by this patch. Moreover, the
arch_get_random_long() loop in crng_reseed() would need to get reorganized
in order to make sure that there will actually be a sufficient number of
successful invocations when writing to the target buffer area following the
bytes obtained from the input_pool.

Thus, in case failing arch_get_random_long()s in combination with
arch_randomness_required set became a problem in the future, it would be
better to improve the error path and simply return the unused entropy
extracted from the input_pool back.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 49 +++++++++++++++++++++++++------------------
 1 file changed, 29 insertions(+), 20 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 424de1565927..7712b4464ef5 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1190,6 +1190,14 @@ static int crng_slow_load(const char *cp, size_t len)
  */
 static int min_crng_reseed_pool_entropy(void)
 {
+	/*
+	 * If there's an architecture provided RNG, use it for
+	 * up to one half of the minimum entropy needed for
+	 * reseeding. That way it won't dominate the entropy
+	 * collected by other means at input_pool.
+	 */
+	if (arch_has_random() || arch_has_random_seed())
+		return 8;
 	return 16;
 }
 
@@ -1197,6 +1205,7 @@ static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
 {
 	unsigned long	flags;
 	int		i, num;
+	bool		arch_randomness_required = false;
 	union {
 		__u8	block[CHACHA_BLOCK_SIZE];
 		__u32	key[8];
@@ -1205,8 +1214,16 @@ static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
 	if (r) {
 		num = extract_entropy(r, &buf, 32,
 				      min_crng_reseed_pool_entropy());
-		if (num == 0)
+		if (num == 0) {
 			return;
+		} else if (num < 16) {
+			/*
+			 * The input_pool did not provide sufficient
+			 * entropy for reseeding and the architecture
+			 * provided RNG will have to make up for it.
+			 */
+			arch_randomness_required = true;
+		}
 	} else {
 		_extract_crng(&primary_crng, buf.block);
 		_crng_backtrack_protect(&primary_crng, buf.block,
@@ -1216,8 +1233,17 @@ static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
 	for (i = 0; i < 8; i++) {
 		unsigned long	rv;
 		if (!arch_get_random_seed_long(&rv) &&
-		    !arch_get_random_long(&rv))
+		    !arch_get_random_long(&rv)) {
+			if (arch_randomness_required) {
+				/*
+				 * The input_pool failed to provide
+				 * sufficient entropy and the arch RNG
+				 * could not make up for that either.
+				 */
+				return;
+			}
 			rv = random_get_entropy();
+		}
 		buf.key[i] ^= rv;
 	}
 
@@ -1522,8 +1548,6 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	cycles_t		cycles = random_get_entropy();
 	__u32			c_high, j_high;
 	__u64			ip;
-	unsigned long		seed;
-	int			credit = 0;
 	bool			reseed;
 	struct queued_entropy	q = { 0 };
 
@@ -1560,26 +1584,11 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	if (!spin_trylock(&r->lock))
 		return;
 
-	/*
-	 * If we have architectural seed generator, produce a seed and
-	 * add it to the pool further below. For the sake of paranoia
-	 * don't let the architectural seed generator dominate the
-	 * input from the interrupt noise.
-	 */
-	credit = !!arch_get_random_long(&seed);
-
 	fast_pool->last = now;
 	fast_pool->count = 0;
 	/* award one bit for the contents of the fast pool */
-	__queue_entropy(r, &q, (credit + 1) << ENTROPY_SHIFT);
+	__queue_entropy(r, &q, 1 << ENTROPY_SHIFT);
 	__mix_pool_bytes(r, &fast_pool->pool, sizeof(fast_pool->pool));
-	if (credit) {
-		/*
-		 * A seed has been obtained from
-		 * arch_get_random_seed_long() above, mix it in.
-		 */
-		__mix_pool_bytes(r, &seed, sizeof(seed));
-	}
 	reseed = __dispatch_queued_entropy_fast(r, &q);
 	spin_unlock(&r->lock);
 	if (reseed)
-- 
2.26.2

[RFC PATCH 22/41] random: introduce arch_has_sp800_90b_random_seed()

[Nicolai Stange]

NIST SP800-90C allows for the combination of multiple SP800-90B entropy
sources by concatenating their individual outputs together, c.f.
sec. 5.3.4 and also the constructions in 10.3.1 from the second draft.

We're already doing exactly that when reseeding the primary CRNG from the
input pool and possibly from arch_get_random_seed_long() +
arch_get_random_long(). The input pool will be moved gradually towards
SP800-90B compliance with future patches. Provide a means for the random
driver to check whether arch_get_random_seed_long() is also a full entropy
source conforming to NIST SP800-90B.

Note that I couldn't find any explicit statement in the specs that would
allow for using NRBGs as defined by SP800-90C as a drop-in replacement for
"entropy sources" in the sense of SP800-90B. In particular there is no
statement that NRBGs may be combined with other SP800-90B entropy sources.
However, NRBGs always provide stronger guarantees in that they provide
certain protection against silent failure of backing entropy sources. Thus,
I think it would be perfectly acceptable to combine SP800-90C NRBGs, i.e.
ARMv8's RNDRRS or x86's RDSEED with other entropy sources.

Introduce arch_has_sp800_90b_random_seed().
- Make the generic stub return false.
- Make the arm64 variant return false as well: the current
  arch_get_random_seed_long() is based on RNDR, not RNDRRS.
- Make it return false on powerpc and s390, too.
- Let arch_has_sp800_90b_random_seed() return true on x86 if the CPU has
  RDSEED support.
Yes, I know, one change per patch, but this is part of a RFC series.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 arch/arm64/include/asm/archrandom.h   | 10 +++++++++-
 arch/powerpc/include/asm/archrandom.h |  5 +++++
 arch/s390/include/asm/archrandom.h    |  5 +++++
 arch/x86/include/asm/archrandom.h     |  8 ++++++++
 include/linux/random.h                |  9 +++++++++
 5 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h
index 055d18713db7..db7813c79b3e 100644
--- a/arch/arm64/include/asm/archrandom.h
+++ b/arch/arm64/include/asm/archrandom.h
@@ -42,7 +42,15 @@ static inline bool arch_has_random_seed(void)
 	 */
 	return cpus_have_const_cap(ARM64_HAS_RNG);
 }
-
+static inine bool arch_has_sp800_90b_random_seed(void)
+{
+	/*
+	 * Section K.12.1 from the Arm Architecture Reference Manual
+	 * Armv8" (DDI0487F) sounds like RNDRRS could qualify as a
+	 * NIST SP800-90C NRBG, but we're currently using RNDR only.
+	 */
+	return false;
+}
 static inline bool __must_check arch_get_random_long(unsigned long *v)
 {
 	return false;
diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h
index 47c2d74e7244..ba0f816d9750 100644
--- a/arch/powerpc/include/asm/archrandom.h
+++ b/arch/powerpc/include/asm/archrandom.h
@@ -16,6 +16,11 @@ static inline bool arch_has_random_seed(void)
 	return ppc_md.get_random_seed;
 }
 
+static inline bool arch_has_sp800_90b_random_seed(void)
+{
+	return false;
+}
+
 static inline bool __must_check arch_get_random_long(unsigned long *v)
 {
 	return false;
diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h
index 18973845634c..1ee7f9e4b255 100644
--- a/arch/s390/include/asm/archrandom.h
+++ b/arch/s390/include/asm/archrandom.h
@@ -31,6 +31,11 @@ static inline bool arch_has_random_seed(void)
 	return static_branch_likely(&s390_arch_random_available);
 }
 
+static inline bool arch_has_sp800_90b_random_seed(void)
+{
+	return false;
+}
+
 static inline bool __must_check arch_get_random_long(unsigned long *v)
 {
 	return false;
diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h
index 030f46c9e310..94d4ee8c9e45 100644
--- a/arch/x86/include/asm/archrandom.h
+++ b/arch/x86/include/asm/archrandom.h
@@ -80,6 +80,14 @@ static inline bool arch_has_random_seed(void)
 	return static_cpu_has(X86_FEATURE_RDSEED);
 }
 
+static inline bool arch_has_sp800_90b_random_seed(void)
+{
+	/*
+	 * According to the Intel SDM, rdseed is NIST SP800-90B
+	 * compliant.
+	 */
+	return arch_has_random_seed();
+}
 static inline bool __must_check arch_get_random_long(unsigned long *v)
 {
 	return arch_has_random() ? rdrand_long(v) : false;
diff --git a/include/linux/random.h b/include/linux/random.h
index d4653422a0c7..933f5daa4a1c 100644
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -128,6 +128,15 @@ static inline bool arch_has_random_seed(void)
 {
 	return false;
 }
+/*
+ * Whether or not arch_get_random_seed_long() qualifies as a NIST
+ * SP800-90B compliant entropy source providing full entropy output.
+ * NIST SP800-90C NRBG's are probably fine, too.
+ */
+static inline bool arch_has_sp800_90b_random_seed(void)
+{
+	return false;
+}
 static inline bool __must_check arch_get_random_long(unsigned long *v)
 {
 	return false;
-- 
2.26.2

[RFC PATCH 23/41] random: don't award entropy to non-SP800-90B arch RNGs in FIPS mode

[Nicolai Stange]

It is required by SP800-90C that only SP800-90B compliant entropy sources
may be used for seeding DRBGs.

Don't award any entropy to arch_get_random_long() if fips_enabled is
true. Don't award any entropy to arch_get_random_seed_long() if
fips_enabled && !arch_has_sp800_90b_random_seed().

This is achieved by making min_crng_reseed_pool_entropy() return the
full minimum seed size if fips_enabled && !arch_has_sp800_90b_random_seed()
is true. This prevents crng_reseed() from attempting to make up for any
lack of entropy in the input_pool by reading from the architectural RNG.

Make crng_reseed() bail out in FIPS mode if the input_pool provides
insufficient entropy and any of the arch_get_random_seed_long()
invocations fails: there's no statement regarding SP900-90B compliance of
arch_get_random_long() and so it can't be used as a backup.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 7712b4464ef5..aaddee4e4ab1 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1195,9 +1195,13 @@ static int min_crng_reseed_pool_entropy(void)
 	 * up to one half of the minimum entropy needed for
 	 * reseeding. That way it won't dominate the entropy
 	 * collected by other means at input_pool.
+	 * If in FIPS mode, restrict this to SP900-90B compliant
+	 * architectural RNGs.
 	 */
-	if (arch_has_random() || arch_has_random_seed())
+	if (arch_has_sp800_90b_random_seed() ||
+	    (!fips_enabled && (arch_has_random() || arch_has_random_seed()))) {
 		return 8;
+	}
 	return 16;
 }
 
@@ -1233,7 +1237,8 @@ static void crng_reseed(struct crng_state *crng, struct entropy_store *r)
 	for (i = 0; i < 8; i++) {
 		unsigned long	rv;
 		if (!arch_get_random_seed_long(&rv) &&
-		    !arch_get_random_long(&rv)) {
+		    ((arch_randomness_required && fips_enabled) ||
+		     !arch_get_random_long(&rv))) {
 			if (arch_randomness_required) {
 				/*
 				 * The input_pool failed to provide
-- 
2.26.2

[RFC PATCH 24/41] init: call time_init() before rand_initialize()

[Nicolai Stange]

Commit d55535232c3d ("random: move rand_initialize() earlier") moved the
rand_initialize() invocation from the early initcalls to right after
timekeeping_init(), but before time_init().

However, rand_initialize() would indirectly invoke random_get_entropy(),
which is an alias to get_cycles() on most archs, in case an architectural
RNG is not available. Problem is that on some archs, e.g. ARM,
get_cycles() can only be relied upon when time_init() has completed.

Move the invocation of time_init() a couple of lines up in start_kernel()
so that it gets called before rand_initialize().

Note that random_get_entropy() data doesn't get any entropy credit and
thus, this issue is not to be considered a bug, but more of an
inconsistency.

Fixes: d55535232c3d ("random: move rand_initialize() earlier")
Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 init/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/init/main.c b/init/main.c
index ae78fb68d231..30892675f48e 100644
--- a/init/main.c
+++ b/init/main.c
@@ -942,6 +942,7 @@ asmlinkage __visible void __init __no_sanitize_address start_kernel(void)
 	hrtimers_init();
 	softirq_init();
 	timekeeping_init();
+	time_init();
 
 	/*
 	 * For best initial stack canary entropy, prepare it after:
@@ -956,7 +957,6 @@ asmlinkage __visible void __init __no_sanitize_address start_kernel(void)
 	add_device_randomness(command_line, strlen(command_line));
 	boot_init_stack_canary();
 
-	time_init();
 	perf_event_init();
 	profile_init();
 	call_function_init();
-- 
2.26.2

[RFC PATCH 25/41] random: probe cycle counter resolution at initialization

[Nicolai Stange]

An upcoming patch will change the entropy estimate per
add_interrupt_randomness() event for fips_enabled based on whether
random_get_entropy() resp. get_cycles() is able to capture individual
instructions.

For example, x86's TSC would qualify, whereas I've seen cycle counters on
e.g. a Raspberry PI 2B with an advertised resolution of only 52ns even
though the CPU had been clocked at 1GHz. And then there's possibly hardware
which doesn't have a cycle counter at all and where get_cycles() would
always return the same constant.

Make rand_initialize() probe the cycle counter resolution.

Introduce a new static_key have_highres_cycle_ctr, indicicating whether
or not the system's cycle counter is able to capture individual
instructions. Initially it's set to true. Introduce
probe_cycle_ctr_resolution() and call it from rand_initialize().
Make probe_cycle_ctr_resolution() compare 16 successive
random_get_entropy() values and disable have_highres_cycle_ctr in case
the same value has been read two times in a row. As have_highres_cycle_ctr
will be only accessed if fips_enabled is true, make it return early in
case it's not set.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index aaddee4e4ab1..a985ceb22c7c 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -335,6 +335,7 @@
 #include <linux/syscalls.h>
 #include <linux/completion.h>
 #include <linux/uuid.h>
+#include <linux/jump_label.h>
 #include <crypto/chacha.h>
 #include <crypto/sha.h>
 
@@ -478,6 +479,8 @@ static struct ratelimit_state urandom_warning =
 
 static int ratelimit_disable __read_mostly;
 
+static DEFINE_STATIC_KEY_TRUE(have_highres_cycle_ctr);
+
 module_param_named(ratelimit_disable, ratelimit_disable, int, 0644);
 MODULE_PARM_DESC(ratelimit_disable, "Disable random ratelimit suppression");
 
@@ -2170,6 +2173,31 @@ static void __init init_std_data(struct entropy_store *r)
 	mix_pool_bytes(r, utsname(), sizeof(*(utsname())));
 }
 
+static void probe_cycle_ctr_resolution(void)
+{
+	cycles_t prev;
+	int i;
+
+	if (!fips_enabled)
+		return;
+
+	/*
+	 * Check if the cycle counter has insn granularity (or at
+	 * least close to).
+	 */
+	prev = random_get_entropy();
+	for (i = 0; i < 16; ++i) {
+		cycles_t next;
+
+		next = random_get_entropy();
+		if (next == prev) {
+			static_branch_disable(&have_highres_cycle_ctr);
+			return;
+		}
+		prev = next;
+	}
+}
+
 /*
  * Note that setup_arch() may call add_device_randomness()
  * long before we get here. This allows seeding of the pools
@@ -2182,6 +2210,7 @@ static void __init init_std_data(struct entropy_store *r)
  */
 int __init rand_initialize(void)
 {
+	probe_cycle_ctr_resolution();
 	init_std_data(&input_pool);
 	crng_initialize_primary(&primary_crng);
 	crng_global_init_time = jiffies;
-- 
2.26.2

[RFC PATCH 26/41] random: implement support for evaluating larger fast_pool entropies

[Nicolai Stange]

The fast_pools, mixed into from add_interrupt_randomness(), are 128 bits
wide and get awarded at most an entropy value as low as one bit in total.

An upcoming patch will significantly increase the estimated entropy per
event and this will make the fast_pools to receive much larger values
of successively mixed in event entropy.

In analogy to the reasoning in commit 30e37ec516ae ("random: account for
entropy loss due to overwrites"), probabilistic collisions will have to get
accounted for when calculating the total fast_pool entropy.

From said commit, the final fast_pool entropy equals

  e = pool_size * (1 - exp(-num_events * entropy_per_event / pool_size))

Where pool_size is 128 bits in this case and
num_events * entropy_per_event equals the sum of all estimated entropy from
the IRQ events previously mixed in.

Disclaimer: I'm cargo-culting here. That probabilisic overwrites are in
fact an issue sounds plausible after having read e.g. "Balls into Bins" by
M.Raab and A. Steger. But I haven't managed to derive this equation by
myself, nor have I found it in any literature.

Anyway, implement the new fast_pool_entropy() for evaluating the equation
given above by means of a suitable approximation.

add_interrupt_randomness() empties its fast_pool into the global input_pool
whenever the number of accumulated events has reached a threshold of 64
and the input_pool's ->lock is uncontended. Thus, the number of IRQ events
accumulated at the fast_pool can be assumed to be unlikely to exceed
larger factors of 64. The maximum estimate supported for per-IRQ entropy
will be 1 bit and thus, this sets an upper bound on the range where the
approximation is supposed to work well.

At the same time, estimates for the per-IRQ entropy as low as 1/64 bits
should be supported and the approximation should not be too coarse in these
lower regions in order to avoid excessive loss when entropy is likely a
scarce resource anyway.

Apply a piecewise linear approximation to the fast_pool entropy, with
the lengths of the resp. intervals getting doubled with increasing input
values. That is, let the first interval cover 32 bits worth of input
entropy, the next one 64 bits and stop after a final one of length
128 bits. Any input entropy beyond 32 + 64 + 128 bits gets discarded in
order to limit the computations done from interrupt context, but as
outlined above, this is unlikely to matter in practice. The shorter
intervals for the regions of smaller values will improve the accuracy of
the approximation in these areas, i.e. for small estimates for the per-IRQ
entropy.

Note that the new fast_pool_entropy() is not being used anywhere yet, it
will be wired up in an upcoming commit.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 52 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index a985ceb22c7c..ac36c56dd135 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1547,6 +1547,58 @@ static __u32 get_reg(struct fast_pool *f, struct pt_regs *regs)
 	return *ptr;
 }
 
+/*
+ * Calculate the entropy of a fast_pool after num_events IRQ events of
+ * assessed entropy 2^-event_entropy_shift each have been mixed in.
+ */
+static unsigned int fast_pool_entropy(unsigned int num_events,
+				      int event_entropy_shift)
+{
+	unsigned int result, cur, interval_len;
+
+	/*
+	 * Assume that each event fed into the fast_pool
+	 * carries h = 2^-event_entropy_shift bits of entropy.
+	 * In analogy to how entropy deltas are calculated
+	 * in pool_entropy_delta() for the struct entropy_store
+	 * input_pool, a fast_pool which received num_events
+	 * of total entropy num_events * h will contain
+	 * p * (1 - exp(-num_events * h / p)
+	 * bits of entropy, where p equals the poolsize of 128 bits.
+	 *
+	 * Note that most of the time num_events will be ~64, c.f.
+	 * add_interrupt_randomness.  Approximate the resulting
+	 * fast_pool entropy in a piecewise linear manner from below:
+	 * from 0 to 32, from 32 to 96 and from 96 to 224.
+	 * Event entropy above 224 gets simply discarded. For p = 128,
+	 * the calculated fast_pool entropy is ~226 at
+	 * num_events * h == 32, ~540 at 96 and ~846 at 224, all given
+	 * in units of 2^-ENTROPY_SHIFT.
+	 */
+	BUILD_BUG_ON(sizeof(((struct fast_pool *)NULL)->pool) != 16);
+	BUILD_BUG_ON(ENTROPY_SHIFT != 3);
+
+	/* Interval from 0 to 32. */
+	interval_len = 32 << event_entropy_shift;
+	cur = min_t(unsigned int, num_events, interval_len);
+	result = (226 * cur) >> 5; /* shift is for /32 */
+	num_events -= cur;
+
+	/* Interval of length 64 from 32 to 96. */
+	interval_len <<= 1;
+	cur = min_t(unsigned int, num_events, interval_len);
+	result += ((540 - 226) * cur) >> 6; /* shift is for /64 */
+	num_events -= cur;
+
+	/* Interval of length 128 from 96 to 224. */
+	interval_len <<= 1;
+	cur = min_t(unsigned int, num_events, interval_len);
+	result += ((846 - 540) * cur) >> 7; /* shift is for /128 */
+
+	/* Return value is in units of 2^-ENTROPY_SHIFT. */
+	return result >> event_entropy_shift;
+}
+
 void add_interrupt_randomness(int irq, int irq_flags)
 {
 	struct entropy_store	*r;
-- 
2.26.2

[RFC PATCH 27/41] random: increase per-IRQ event entropy estimate if in FIPS mode

[Nicolai Stange]

NIST SP800-90C prohibits the use of multiple correlated entropy sources.
However, add_interrupt_randomness(), add_disk_randomness() and
add_input_randomness() are clearly not independent and an upcoming patch
will make the latter two to stop contributing any entropy to the global
balance if fips_enabled is on.

With the current parameter settings, it can be assumed that
add_disk_randomness() resp. add_input_randomness() are the dominating
contributors to the overall entropy reserve for some common workloads: both
more or less estimate the entropy per event to equal the width of the
minimum out of the first, second and third jiffes deltas to the previous
occurrence.

add_interrupt_randomness() on the other hand only attributes one single bit
entropy to a full batch of 64 IRQ events (or once a second if that
completes earlier).

Thus, the upcoming exclusion of two potent entropy sources should somehow
be compensated for.

Stephan Müller worked around this very problem in his "LRNG" proposal ([1])
by increasing the entropy estimate per IRQ event. Namely, in case a
get_cycles() with instruction granularity is available, he estimated one
bit of entropy per IRQ event and (IIRC) 1/10 bits otherwise. I haven't
tested this claim myself, in particular not on smaller devices. But for the
sake of moving the development of this RFC series forward, I'll assume it
as granted and hereby postulate that

  The lower eight bits of the differences between get_cycles() from two
  successive IRQ events on the same CPU carry
  - one bit of min-entropy in case a get_cycles() with instruction
    granularity is available and
  - 1/8 bit of min-entropy in case get_cycles() is still non-trivial, but
    has a lower resolution.

In particular this is assumed to be true for highly periodic interrupts
like those issued for e.g. USB microframes and on all supported
architectures. In the former case, the underlying source of randomness is
believed to follow the same principles as for the Jitter RNGs resp.
try_to_generate_entropy(): diffences in RAM vs. CPU clockings and
unpredictability of cache states to a certain extent.

Notes:
- NIST SP800-90B requires a means to access raw samples for validation
  purposes. Implementation of such an interface is deliberately not part
  of this RFC series here, but would necessarily be subject of future work.
  So there would be a means to at least validate these assumptions.
- The choice of 1/8 over the 1/10 from the LRNG patchset has been made
  because it's a power of two and I suppose that the estimate of 1/10
  had been quite arbitrary anyway. Replacement of the 1/8 by smaller
  powers of two down to 1/64 will be supported throughout this patch
  series.

Some health tests as required by NIST SP800-90B will be implemented later
in this series. In order to allow for dynamically decreasing the assessed
entropy on a per-CPU basis upon health test failures, make it an attibute
of the per-CPU struct fast_pool. That is, introduce a new integer field
->event_entropy_shift to struct fast_pool. The estimated entropy per
IRQ sample will be calculated as 2^-event_entropy_shift. Initialize it
statically with -1 to indicate that runtime initialization hasn't happened
yet. Introduce fast_pool_init_accounting() which gets called
unconditionally from add_interrupt_randomness() for doing the necessary
runtime initializations once, i.e. if ->event_entropy_shift is
still found to be negative. Implement it with the help of the also new
min_irq_event_entropy_shift(), which will return the initial
->event_entropy_shift value as determined according to the rules from
above. That is, depending on the have_highres_cycle_ctr, the result is
eiher zero or three. Note that have_highres_cycle_ctr will only get
properly initialized from rand_initialize() if fips_enabled is set, but
->event_entropy_shift will also only ever get accessed in this case.

Finally, for the case tha fips_enabled is set, make
add_interrupt_randomness() to estimate the amount of entropy transferred
from the fast_pool into the global input_pool as
fast_pool_entropy(->count, ->event_entropy_shift), rather than only one
single bit. Remember that fast_pool_entropy() calculates the amount of
entropy contained in a fast_pool, based on the total number of events mixed
into it and the estimated entropy per event.

[1] https://lkml.kernel.org/r/5695397.lOV4Wx5bFT@positron.chronox.de

Suggested-by: Stephan Müller <smueller@chronox.de>
Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 50 ++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 45 insertions(+), 5 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index ac36c56dd135..8f79e90f2429 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -615,6 +615,7 @@ struct fast_pool {
 	unsigned long	last;
 	unsigned short	reg_idx;
 	unsigned char	count;
+	int		event_entropy_shift;
 };
 
 /*
@@ -1509,7 +1510,9 @@ void add_input_randomness(unsigned int type, unsigned int code,
 }
 EXPORT_SYMBOL_GPL(add_input_randomness);
 
-static DEFINE_PER_CPU(struct fast_pool, irq_randomness);
+static DEFINE_PER_CPU(struct fast_pool, irq_randomness) = {
+	.event_entropy_shift = -1,
+};
 
 #ifdef ADD_INTERRUPT_BENCH
 static unsigned long avg_cycles, avg_deviation;
@@ -1599,6 +1602,32 @@ static unsigned int fast_pool_entropy(unsigned int num_events,
 	return result >> event_entropy_shift;
 }
 
+static inline int min_irq_event_entropy_shift(void)
+{
+	if (static_branch_likely(&have_highres_cycle_ctr)) {
+		/*
+		 * If a cycle counter with a good enough resolution is
+		 * available, estimate the entropy per IRQ event to
+		 * be no more than 2^-0 == 1 bit.
+		 */
+		return 0;
+	}
+
+	/*
+	 * Otherwise return an estimate upper bound of
+	 * 2^-3 == 1/8 bit per event.
+	 */
+	return 3;
+}
+
+static inline void fast_pool_init_accounting(struct fast_pool *f)
+{
+	if (likely(f->event_entropy_shift >= 0))
+		return;
+
+	f->event_entropy_shift = min_irq_event_entropy_shift();
+}
+
 void add_interrupt_randomness(int irq, int irq_flags)
 {
 	struct entropy_store	*r;
@@ -1610,6 +1639,7 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	__u64			ip;
 	bool			reseed;
 	struct queued_entropy	q = { 0 };
+	unsigned int		nfrac;
 
 	if (cycles == 0)
 		cycles = get_reg(fast_pool, regs);
@@ -1644,13 +1674,23 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	if (!spin_trylock(&r->lock))
 		return;
 
-	fast_pool->last = now;
-	fast_pool->count = 0;
-	/* award one bit for the contents of the fast pool */
-	__queue_entropy(r, &q, 1 << ENTROPY_SHIFT);
+	fast_pool_init_accounting(fast_pool);
+
+	if (!fips_enabled) {
+		/* award one bit for the contents of the fast pool */
+		nfrac = 1 << ENTROPY_SHIFT;
+	} else {
+		nfrac = fast_pool_entropy(fast_pool->count,
+					  fast_pool->event_entropy_shift);
+	}
+	__queue_entropy(r, &q, nfrac);
 	__mix_pool_bytes(r, &fast_pool->pool, sizeof(fast_pool->pool));
 	reseed = __dispatch_queued_entropy_fast(r, &q);
 	spin_unlock(&r->lock);
+
+	fast_pool->last = now;
+	fast_pool->count = 0;
+
 	if (reseed)
 		crng_reseed(&primary_crng, r);
 }
-- 
2.26.2

[RFC PATCH 28/41] random: don't award entropy to disk + input events if in FIPS mode

[Nicolai Stange]

NIST SP800-90C prohibits the use of multiple correlated entropy sources.

Obviously, add_disk_randomness(), add_input_randomness() and
add_interrupt_randomness() are not independent.

Follow the approach taken by Stephan Müller's LRNG patchset ([1]) and don't
award any entropy to the former two if fips_enabled is true.
Note that the entropy loss has already been compensated for by a previous
patch increasing the IRQ event estimate.

The actual entropy accounting from add_disk_randomness() and
add_input_randomness() is implemented in the common add_timer_randomness()
called therefrom.

Make the latter to not dispatch any entropy to the global entropy balance
if fips_enabled is on.

[1] https://lkml.kernel.org/r/5695397.lOV4Wx5bFT@positron.chronox.de

Suggested-by: Stephan Müller <smueller@chronox.de>
Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 8f79e90f2429..680ccc82a436 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1481,12 +1481,24 @@ static void add_timer_randomness(struct timer_rand_state *state, unsigned num)
 
 	r = &input_pool;
 	spin_lock_irqsave(&r->lock, flags);
-	/*
-	 * delta is now minimum absolute delta.
-	 * Round down by 1 bit on general principles,
-	 * and limit entropy estimate to 12 bits.
-	 */
-	__queue_entropy(r, &q, min_t(int, fls(delta>>1), 11) << ENTROPY_SHIFT);
+	if (!fips_enabled) {
+		unsigned int nfrac;
+
+		/*
+		 * delta is now minimum absolute delta.
+		 * Round down by 1 bit on general principles,
+		 * and limit entropy estimate to 12 bits.
+		 */
+		nfrac = min_t(int, fls(delta>>1), 11) << ENTROPY_SHIFT;
+		__queue_entropy(r, &q, nfrac);
+	} else {
+		/*
+		 * Multiple correlated entropy sources are prohibited
+		 * by NIST SP800-90C. Leave it up to
+		 * add_interrupt_randomness() to contribute any
+		 * entropy.
+		 */
+	}
 	__mix_pool_bytes(r, &sample, sizeof(sample));
 	reseed = __dispatch_queued_entropy_fast(r, &q);
 	spin_unlock_irqrestore(&r->lock, flags);
-- 
2.26.2

[RFC PATCH 29/41] random: move definition of struct queued_entropy and related API upwards

[Nicolai Stange]

The next patch will add a member of type struct queued_entropy to struct
fast_pool and thus, the former's definition needs to be visible at the
latter's.

Move the definition of struct queued_entropy upwards in the file so that
it comes before the definition struct fast_pool. Move the associated
function definitions as well in order to keep everything together.

Note that said function definitions had originally been inserted at the
old location with the intent to minimize their introducing patch's diff
by placing them near the now removed credit_entropy_delta() they
superseded.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 243 +++++++++++++++++++++---------------------
 1 file changed, 124 insertions(+), 119 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 680ccc82a436..55e784a5a2ec 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -519,6 +519,10 @@ static ssize_t extract_entropy(struct entropy_store *r, void *buf,
 static ssize_t _extract_entropy(struct entropy_store *r, void *buf,
 				size_t nbytes, int fips);
 
+static unsigned int pool_entropy_delta(struct entropy_store *r,
+				       int base_entropy_count,
+				       int nfrac, bool fast);
+
 static int min_crng_reseed_pool_entropy(void);
 
 static void crng_reseed(struct crng_state *crng, struct entropy_store *r);
@@ -610,125 +614,6 @@ static void mix_pool_bytes(struct entropy_store *r, const void *in,
 	spin_unlock_irqrestore(&r->lock, flags);
 }
 
-struct fast_pool {
-	__u32		pool[4];
-	unsigned long	last;
-	unsigned short	reg_idx;
-	unsigned char	count;
-	int		event_entropy_shift;
-};
-
-/*
- * This is a fast mixing routine used by the interrupt randomness
- * collector.  It's hardcoded for an 128 bit pool and assumes that any
- * locks that might be needed are taken by the caller.
- */
-static void fast_mix(struct fast_pool *f)
-{
-	__u32 a = f->pool[0],	b = f->pool[1];
-	__u32 c = f->pool[2],	d = f->pool[3];
-
-	a += b;			c += d;
-	b = rol32(b, 6);	d = rol32(d, 27);
-	d ^= a;			b ^= c;
-
-	a += b;			c += d;
-	b = rol32(b, 16);	d = rol32(d, 14);
-	d ^= a;			b ^= c;
-
-	a += b;			c += d;
-	b = rol32(b, 6);	d = rol32(d, 27);
-	d ^= a;			b ^= c;
-
-	a += b;			c += d;
-	b = rol32(b, 16);	d = rol32(d, 14);
-	d ^= a;			b ^= c;
-
-	f->pool[0] = a;  f->pool[1] = b;
-	f->pool[2] = c;  f->pool[3] = d;
-	f->count++;
-}
-
-static void process_random_ready_list(void)
-{
-	unsigned long flags;
-	struct random_ready_callback *rdy, *tmp;
-
-	spin_lock_irqsave(&random_ready_list_lock, flags);
-	list_for_each_entry_safe(rdy, tmp, &random_ready_list, list) {
-		struct module *owner = rdy->owner;
-
-		list_del_init(&rdy->list);
-		rdy->func(rdy);
-		module_put(owner);
-	}
-	spin_unlock_irqrestore(&random_ready_list_lock, flags);
-}
-
-/*
- * Based on the pool's current entropy fill level, specified as
- * base_entropy_count, and the number of new entropy bits in units of
- * 2^-ENTROPY_SHIFT to add, return the amount of new entropy to
- * credit. If the 'fast' parameter is set to true, the calculation
- * will be guaranteed to terminate quickly, but this comes at the
- * expense of capping nbits to one half of the pool size.
- */
-static unsigned int pool_entropy_delta(struct entropy_store *r,
-				       int base_entropy_count,
-				       int nfrac, bool fast)
-{
-	const int pool_size = r->poolinfo->poolfracbits;
-	int entropy_count = base_entropy_count;
-
-	if (!nfrac)
-		return 0;
-
-	if (pool_size <= base_entropy_count)
-		return 0;
-
-	/*
-	 * Credit: we have to account for the possibility of
-	 * overwriting already present entropy.	 Even in the
-	 * ideal case of pure Shannon entropy, new contributions
-	 * approach the full value asymptotically:
-	 *
-	 * entropy <- entropy + (pool_size - entropy) *
-	 *	(1 - exp(-add_entropy/pool_size))
-	 *
-	 * For add_entropy <= pool_size/2 then
-	 * (1 - exp(-add_entropy/pool_size)) >=
-	 *    (add_entropy/pool_size)*0.7869...
-	 * so we can approximate the exponential with
-	 * 3/4*add_entropy/pool_size and still be on the
-	 * safe side by adding at most pool_size/2 at a time.
-	 *
-	 * The use of pool_size-2 in the while statement is to
-	 * prevent rounding artifacts from making the loop
-	 * arbitrarily long; this limits the loop to log2(pool_size)*2
-	 * turns no matter how large nbits is.
-	 */
-	do {
-		/* The +2 corresponds to the /4 in the denominator */
-		const int s = r->poolinfo->poolbitshift + ENTROPY_SHIFT + 2;
-		unsigned int anfrac = min(nfrac, pool_size/2);
-		unsigned int add =
-			((pool_size - entropy_count)*anfrac*3) >> s;
-
-		entropy_count += add;
-		nfrac -= anfrac;
-	} while (unlikely(!fast && entropy_count < pool_size-2 && nfrac));
-
-	if (WARN_ON(entropy_count < 0)) {
-		pr_warn("negative entropy/overflow: pool %s count %d\n",
-			r->name, entropy_count);
-		entropy_count = base_entropy_count;
-	} else if (entropy_count > pool_size) {
-		entropy_count = pool_size;
-	}
-
-	return entropy_count - base_entropy_count;
-}
-
 struct queued_entropy {
 	unsigned int pool_watermark_seq;
 	unsigned int queued_entropy_fracbits;
@@ -994,6 +879,126 @@ static void discard_queued_entropy(struct entropy_store *r,
 	spin_unlock_irqrestore(&r->lock, flags);
 }
 
+struct fast_pool {
+	__u32		pool[4];
+	unsigned long	last;
+	unsigned short	reg_idx;
+	unsigned char	count;
+	int		event_entropy_shift;
+};
+
+/*
+ * This is a fast mixing routine used by the interrupt randomness
+ * collector.  It's hardcoded for an 128 bit pool and assumes that any
+ * locks that might be needed are taken by the caller.
+ */
+static void fast_mix(struct fast_pool *f)
+{
+	__u32 a = f->pool[0],	b = f->pool[1];
+	__u32 c = f->pool[2],	d = f->pool[3];
+
+	a += b;			c += d;
+	b = rol32(b, 6);	d = rol32(d, 27);
+	d ^= a;			b ^= c;
+
+	a += b;			c += d;
+	b = rol32(b, 16);	d = rol32(d, 14);
+	d ^= a;			b ^= c;
+
+	a += b;			c += d;
+	b = rol32(b, 6);	d = rol32(d, 27);
+	d ^= a;			b ^= c;
+
+	a += b;			c += d;
+	b = rol32(b, 16);	d = rol32(d, 14);
+	d ^= a;			b ^= c;
+
+	f->pool[0] = a;  f->pool[1] = b;
+	f->pool[2] = c;  f->pool[3] = d;
+	f->count++;
+}
+
+static void process_random_ready_list(void)
+{
+	unsigned long flags;
+	struct random_ready_callback *rdy, *tmp;
+
+	spin_lock_irqsave(&random_ready_list_lock, flags);
+	list_for_each_entry_safe(rdy, tmp, &random_ready_list, list) {
+		struct module *owner = rdy->owner;
+
+		list_del_init(&rdy->list);
+		rdy->func(rdy);
+		module_put(owner);
+	}
+	spin_unlock_irqrestore(&random_ready_list_lock, flags);
+}
+
+/*
+ * Based on the pool's current entropy fill level, specified as
+ * base_entropy_count, and the number of new entropy bits in units of
+ * 2^-ENTROPY_SHIFT to add, return the amount of new entropy to
+ * credit. If the 'fast' parameter is set to true, the calculation
+ * will be guaranteed to terminate quickly, but this comes at the
+ * expense of capping nbits to one half of the pool size.
+ */
+static unsigned int pool_entropy_delta(struct entropy_store *r,
+				       int base_entropy_count,
+				       int nfrac, bool fast)
+{
+	const int pool_size = r->poolinfo->poolfracbits;
+	int entropy_count = base_entropy_count;
+
+	if (!nfrac)
+		return 0;
+
+	if (pool_size <= base_entropy_count)
+		return 0;
+
+	/*
+	 * Credit: we have to account for the possibility of
+	 * overwriting already present entropy.	 Even in the
+	 * ideal case of pure Shannon entropy, new contributions
+	 * approach the full value asymptotically:
+	 *
+	 * entropy <- entropy + (pool_size - entropy) *
+	 *	(1 - exp(-add_entropy/pool_size))
+	 *
+	 * For add_entropy <= pool_size/2 then
+	 * (1 - exp(-add_entropy/pool_size)) >=
+	 *    (add_entropy/pool_size)*0.7869...
+	 * so we can approximate the exponential with
+	 * 3/4*add_entropy/pool_size and still be on the
+	 * safe side by adding at most pool_size/2 at a time.
+	 *
+	 * The use of pool_size-2 in the while statement is to
+	 * prevent rounding artifacts from making the loop
+	 * arbitrarily long; this limits the loop to log2(pool_size)*2
+	 * turns no matter how large nbits is.
+	 */
+	do {
+		/* The +2 corresponds to the /4 in the denominator */
+		const int s = r->poolinfo->poolbitshift + ENTROPY_SHIFT + 2;
+		unsigned int anfrac = min(nfrac, pool_size/2);
+		unsigned int add =
+			((pool_size - entropy_count)*anfrac*3) >> s;
+
+		entropy_count += add;
+		nfrac -= anfrac;
+	} while (unlikely(!fast && entropy_count < pool_size-2 && nfrac));
+
+	if (WARN_ON(entropy_count < 0)) {
+		pr_warn("negative entropy/overflow: pool %s count %d\n",
+			r->name, entropy_count);
+		entropy_count = base_entropy_count;
+	} else if (entropy_count > pool_size) {
+		entropy_count = pool_size;
+	}
+
+	return entropy_count - base_entropy_count;
+}
+
+
 /*********************************************************************
  *
  * CRNG using CHACHA20
-- 
2.26.2

[RFC PATCH 30/41] random: add a queued_entropy instance to struct fast_pool

[Nicolai Stange]

When health tests are introduced with upcoming patches, it will become
necessary to keep entropy queued across add_interrupt_randomness()
invocations for later dispatch to the global balance.

Prepare for this by adding a struct queued_entropy member to the per-CPU
fast_pool. Use it in place of that queue with automatic storage duration
in add_interrupt_randomness().

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 55e784a5a2ec..37746df53acf 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -885,6 +885,7 @@ struct fast_pool {
 	unsigned short	reg_idx;
 	unsigned char	count;
 	int		event_entropy_shift;
+	struct queued_entropy	q;
 };
 
 /*
@@ -1655,7 +1656,7 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	__u32			c_high, j_high;
 	__u64			ip;
 	bool			reseed;
-	struct queued_entropy	q = { 0 };
+	struct queued_entropy	*q = &fast_pool->q;
 	unsigned int		nfrac;
 
 	if (cycles == 0)
@@ -1700,9 +1701,9 @@ void add_interrupt_randomness(int irq, int irq_flags)
 		nfrac = fast_pool_entropy(fast_pool->count,
 					  fast_pool->event_entropy_shift);
 	}
-	__queue_entropy(r, &q, nfrac);
+	__queue_entropy(r, q, nfrac);
 	__mix_pool_bytes(r, &fast_pool->pool, sizeof(fast_pool->pool));
-	reseed = __dispatch_queued_entropy_fast(r, &q);
+	reseed = __dispatch_queued_entropy_fast(r, q);
 	spin_unlock(&r->lock);
 
 	fast_pool->last = now;
-- 
2.26.2

[RFC PATCH 31/41] random: introduce struct health_test + health_test_reset() placeholders

[Nicolai Stange]

The to be implemented health tests will maintain some per-CPU state as they
successively process the IRQ samples fed into the resp. fast_pool from
add_interrupt_randomness().

In order to not to clutter future patches with trivialities, introduce
an empty struct health_test supposed to keep said state in the future.
Add a member of this new type to struct fast_pool.

Introduce a health_test_reset() stub, which is supposed to (re)initialize
instances of struct health_test.

Invoke it from the fast_pool_init_accounting() to make sure that a
fast_pool's contained health_test instance gets initialized once before
its first usage.

Make add_interrupt_randomness call fast_pool_init_accounting() earlier:
health test functionality will get invoked before the latter's old location
and it must have been initialized by that time.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 37746df53acf..0f56c873a501 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -879,6 +879,11 @@ static void discard_queued_entropy(struct entropy_store *r,
 	spin_unlock_irqrestore(&r->lock, flags);
 }
 
+struct health_test {};
+
+static void health_test_reset(struct health_test *h)
+{}
+
 struct fast_pool {
 	__u32		pool[4];
 	unsigned long	last;
@@ -886,6 +891,7 @@ struct fast_pool {
 	unsigned char	count;
 	int		event_entropy_shift;
 	struct queued_entropy	q;
+	struct health_test	health;
 };
 
 /*
@@ -1644,6 +1650,7 @@ static inline void fast_pool_init_accounting(struct fast_pool *f)
 		return;
 
 	f->event_entropy_shift = min_irq_event_entropy_shift();
+	health_test_reset(&f->health);
 }
 
 void add_interrupt_randomness(int irq, int irq_flags)
@@ -1674,6 +1681,8 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	add_interrupt_bench(cycles);
 	this_cpu_add(net_rand_state.s1, fast_pool->pool[cycles & 3]);
 
+	fast_pool_init_accounting(fast_pool);
+
 	if (unlikely(crng_init == 0)) {
 		if ((fast_pool->count >= 64) &&
 		    crng_fast_load((char *) fast_pool->pool,
@@ -1692,8 +1701,6 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	if (!spin_trylock(&r->lock))
 		return;
 
-	fast_pool_init_accounting(fast_pool);
-
 	if (!fips_enabled) {
 		/* award one bit for the contents of the fast pool */
 		nfrac = 1 << ENTROPY_SHIFT;
-- 
2.26.2

[RFC PATCH 32/41] random: introduce health test stub and wire it up

[Nicolai Stange]

NIST SP800-90B requires certain statistical tests to be run continuously on
a noise source's output.

In preparation to implementing those, introduce an empty stub,
health_test_process() and wire it up to add_interrupt_randomness(). This
patch does not implement any actual testing functionality yet, it's mereley
meant to define the interactions between add_interrupt_randomness() and
the health tests.

health_test_process() is to be invoked on individual noise samples, i.e.
cycle counter values and returns, either of three possible status
codes indicating to the calling add_interrupt_randomness() that
- either some more samples are needed in order to complete the statistical
  tests,
- that the tests have finished with positive result on the latest run
  of noise samples or
- that the tests have failed.

Introduce an enum health_result defining constants corresponding to these
resp. cases: health_queue, health_dispatch and health_discard. Provide
another value, health_none, to indicate the case that the health tests
are disabled, because e.g. fips_enabled is unset. Make the stub
health_test_process() return this value for now.

As long as the statistical tests need more input noise samples before
reaching a conclusion, health_queue will get returned from
health_test_process(). FWIW, the number of successive input samples needed
by the tests will be at the order of 128 to 8192, depending on the per-IRQ
entropy estimate. add_interrupt_randomness() currently attempts to transfer
the noise kept within in the per-CPU fast_pool, which is of limited
capacity, to the global input_pool as soon as a threshold of 64 events is
reached and it will continue to do so. However, as long as some tests are
pending, i.e. keep returning health_queue, the associated amount of
estimated entropy must not get added to the global input_pool balance, but
queued up at the fast_pool's queued_entropy instance. Once the health test
have eventually succeeded, as indiciated by health_test_process(), the
entropy previously queued up may get dispatched to the global reserve.
OTOH, on test failure health_discard will get returned and all entropy
queued up from add_interrupt_randomness() since the last dispatch (or
discard resp.) must get discarded.

Note that add_interrupt_randomness() will continue to unconditionally mix
the samples into the fast_pools and eventually into the global input_pool
-- the health test results really only affect the entropy accounting.

So, make add_interrupt_randomness() invoke health_test_process() on
the current cycle counter value in case fips_enabled is set.

In case a fast_pool's fill level threshold of 64 events is reached at a
time when health tests are still pending and keep returning health_queue,
let add_interrupt_randomness() continue to mix the fast_pool's contents
into the input_pool as before, but enqueue the associated amount of entropy
at the fast_pool's associated queued_entropy instance for later dispatch.

Both, entropy dispatch as well as discard operations, require a call to
__dequeue_entropy(), which in turn must only get invoked with the
input_pool's ->lock being held. It follows that in case the spin_trylock()
in add_interrupt_randomness() failed, the latter would not be able to
perform entropy dispatch or discard operations immediately at the time
those have been requested by the health tests. Add two new boolean flags,
->dispatch_needed and ->discard_needed, to struct fast_pool. Set them from
add_interrupt_randomness() in case health_test_process() returned
health_dispatch or health_discard resp.. Make the current and subsequent
add_interrupt_randomness() invocations to check for ->dispatch_needed and
->discard_needed and to attempt to execute any pending dispatch/discard
request. Clear ->dispatch_needed and ->discard_needed again when the
prerequisite ->lock could eventually be obtained.

As actual health tests returning anything but health_none haven't been
implemented yet, there is no behavioural change at this point.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 78 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 75 insertions(+), 3 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 0f56c873a501..cb6441b96b8e 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -881,14 +881,30 @@ static void discard_queued_entropy(struct entropy_store *r,
 
 struct health_test {};
 
+enum health_result {
+	health_none,
+	health_queue,
+	health_dispatch,
+	health_discard,
+};
+
 static void health_test_reset(struct health_test *h)
 {}
 
+static enum health_result
+health_test_process(struct health_test *h, unsigned int event_entropy_shift,
+		    u8 sample)
+{
+	return health_none;
+}
+
 struct fast_pool {
 	__u32		pool[4];
 	unsigned long	last;
 	unsigned short	reg_idx;
 	unsigned char	count;
+	bool		dispatch_needed : 1;
+	bool		discard_needed : 1;
 	int		event_entropy_shift;
 	struct queued_entropy	q;
 	struct health_test	health;
@@ -1662,9 +1678,10 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	cycles_t		cycles = random_get_entropy();
 	__u32			c_high, j_high;
 	__u64			ip;
-	bool			reseed;
+	bool			reseed = false;
 	struct queued_entropy	*q = &fast_pool->q;
 	unsigned int		nfrac;
+	enum health_result	health_result = health_none;
 
 	if (cycles == 0)
 		cycles = get_reg(fast_pool, regs);
@@ -1682,6 +1699,12 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	this_cpu_add(net_rand_state.s1, fast_pool->pool[cycles & 3]);
 
 	fast_pool_init_accounting(fast_pool);
+	if (fips_enabled) {
+		health_result =
+			health_test_process(&fast_pool->health,
+					    fast_pool->event_entropy_shift,
+					    cycles);
+	}
 
 	if (unlikely(crng_init == 0)) {
 		if ((fast_pool->count >= 64) &&
@@ -1693,8 +1716,48 @@ void add_interrupt_randomness(int irq, int irq_flags)
 		return;
 	}
 
+	switch (health_result) {
+	case health_dispatch:
+		/*
+		 * Still haven't made it around processing a previous
+		 * entropy discard request?
+		 */
+		fast_pool->dispatch_needed = !fast_pool->discard_needed;
+		break;
+
+	case health_discard:
+		/*
+		 * Still haven't made it around processing a previous
+		 * entropy dispatch request?
+		 */
+		fast_pool->discard_needed = !fast_pool->dispatch_needed;
+		break;
+
+	case health_queue:
+		/*
+		 * If a previous sample triggered a dispatch which is
+		 * still pending, it's impossible to add new events on
+		 * top as far as entropy accounting is
+		 * concerned. Don't count any events until we get a
+		 * hold of the input_pool ->lock and complete the
+		 * dispatch below. Undo the increment from fast_mix()
+		 * above.
+		 */
+		if (fast_pool->dispatch_needed)
+			fast_pool->count--;
+		break;
+
+	case health_none:
+		/*
+		 * fips_enabled is unset, suppress compiler warnings.
+		 */
+		break;
+	};
+
 	if ((fast_pool->count < 64) &&
-	    !time_after(now, fast_pool->last + HZ))
+	    !(health_result == health_none &&
+	      time_after(now, fast_pool->last + HZ)) &&
+	    !fast_pool->dispatch_needed && !fast_pool->discard_needed)
 		return;
 
 	r = &input_pool;
@@ -1710,7 +1773,16 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	}
 	__queue_entropy(r, q, nfrac);
 	__mix_pool_bytes(r, &fast_pool->pool, sizeof(fast_pool->pool));
-	reseed = __dispatch_queued_entropy_fast(r, q);
+
+	if (fast_pool->dispatch_needed || health_result == health_none) {
+		reseed = __dispatch_queued_entropy_fast(r, q);
+		fast_pool->dispatch_needed = false;
+	} else if (fast_pool->discard_needed) {
+		int dummy;
+
+		__dequeue_entropy(r, q, &dummy);
+		fast_pool->discard_needed = false;
+	}
 	spin_unlock(&r->lock);
 
 	fast_pool->last = now;
-- 
2.26.2

[RFC PATCH 33/41] random: make health_test_process() maintain the get_cycles() delta

[Nicolai Stange]

The min-entropy estimate has been made for the lower eight bits of the
deltas between cycle counter values from successive IRQ events and thus,
the upcoming health tests should actually be run on these deltas.

Introduce a new field ->previous_sample to struct health_test for storing
the previous get_cycles() value. Make health_test_process() maintain it
and also calculate the delta between the current and the previous value
at this point already in preparation to passing it to the upcoming health
tests. Note that ->previous_sample is deliberately not touched from
health_test_reset() in order to maintain a steady flow of correctly
calculated deltas across health test resets.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index cb6441b96b8e..33f9b7b59f92 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -879,7 +879,9 @@ static void discard_queued_entropy(struct entropy_store *r,
 	spin_unlock_irqrestore(&r->lock, flags);
 }
 
-struct health_test {};
+struct health_test {
+	u8 previous_sample;
+};
 
 enum health_result {
 	health_none,
@@ -895,6 +897,16 @@ static enum health_result
 health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 		    u8 sample)
 {
+	u8 sample_delta;
+
+	/*
+	 * The min-entropy estimate has been made for the lower eight
+	 * bits of the deltas between cycle counter values from
+	 * successive IRQ events.
+	 */
+	sample_delta = sample - h->previous_sample;
+	h->previous_sample = sample;
+
 	return health_none;
 }
 
-- 
2.26.2

[RFC PATCH 34/41] random: implement the "Adaptive Proportion" NIST SP800-90B health test

[Nicolai Stange]

NIST SP800-90B requires an implementation of the "Adaptive Proportion"
health test (APT) or similar for detecting noise source entropy
degradations.

This tests works by counting how many times the first sample value in a
sequence of n events occurs among the remaining n-1 samples. The test will
reject if this number exceeds a certain threshold.

With a min-entropy estimate of H=2^-event_entropy_shift per IRQ event, the
probability of observing any particular sample value is bounded by
p <= 2^-H. Assuming i.i.d., the number of occurences of such a sample
value among n - 1 events follows the binomial distribution with parameters
n - 1 and p. The probability to observe up to k occurences of a given
sample value is not less than that distribution's CDF F(n - 1, p, k) at
point k, per the definition of CDFs and the fact that
F(n - 1, p1, k) >= F(n - 1, p2, k) for p1 <= p2 in the particular case of
Binomial distributions. It follows that an upper bound on the probability
of observing the same value c or more times among n - 1 consecutive samples
is given by 1 - F(n - 1, p, c - 1). In conclusion, the probability of false
positives is <= p * (1 - F(n - 1, p, c - 1)) for the Adaptive Proportion
test.

NIST SP800-90B recommends to set n to either 512 or 1024 and to choose a
cut-off value c such that the probability of false positives is <= 2^-20.
However, assuming an estimated per-IRQ entropy of 1 bit, it would take
1024/128 == 8 minimum crng seed sizes worth of entropy before the APT
eventually completes and the accumulated entropy may get released to the
global reserve. Thus, it is desirable to set n such that the APT will
complete within 128 bits worth of entropy, i.e. to n = 128 / H. However,
for such relatively small values of n, an upper bound as small as 2^-20
for the false positives probability would make the test's statistical
power, i.e. the capability to detect degraded noise sources, plummet to
uselessness. Note that add_interrupt_randomness() will continue to
unconditionally mix all events into the fast_pools, independent of the
APT's outcome. Thus, allowing for a higher probability of false positives
cannot change the output distribution, but only potentially affect the
entropy accounting. Choose an upper bound of 2^-16 for the probability of
false positives.

The resulting cut-off values for the different supported values of per-IRQ
entropy estimates are tabulated below. The "power" column lists the
probabilities (again for i.i.d.) that the APT would report a failure in
case the actual entropy has degraded to one half of the assumed estimate.

   H     n   c    power
   --------------------
      1  128   87 52.5%
    1/2  256  210 67.5%
    1/4  512  463 76.7%
    1/8 1024  973 82.8%
   1/16 2048 1997 82.6%
   1/32 4096 4044 85.8%
   1/64 8192 8140 85.8%

Add a couple of new fields to struct health_test for storing the required
APT state to struct health_test:
 - ->apt_event_count: total number of samples processed by the currently
     pending APT,
 - ->apt_candidate: the sample value whose number of occurences the
     currently pending APT is counting,
 - ->apt_candidate_count: the number of occurences of ->apt_candidate
     the currently pending APT has encountered so far.

Implement the APT logic and wrap it in a new function, health_test_apt().
Invoke it from health_test_process().

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 56 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 54 insertions(+), 2 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 33f9b7b59f92..131302cbc495 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -880,6 +880,10 @@ static void discard_queued_entropy(struct entropy_store *r,
 }
 
 struct health_test {
+	unsigned short apt_event_count;
+	unsigned short apt_candidate_count;
+	u8 apt_candidate;
+
 	u8 previous_sample;
 };
 
@@ -890,8 +894,56 @@ enum health_result {
 	health_discard,
 };
 
+/* Adaptive Proportion Test */
+static void health_apt_reset(struct health_test *h)
+{
+	h->apt_event_count = 0;
+}
+
+static enum health_result
+health_test_apt(struct health_test *h, unsigned int event_entropy_shift,
+		u8 sample_delta)
+{
+	unsigned int n = 128 << event_entropy_shift;
+	/*
+	 * Observing some particular sample value more often than
+	 * these thresholds, specified for the different possible
+	 * values of event_entropy_shift each, should have probability
+	 * <= 2^-16.
+	 */
+	static const unsigned int c[] = {87, 210, 463, 973, 1997, 4044, 8140};
+
+	if (!h->apt_event_count) {
+		h->apt_event_count = 1;
+		h->apt_candidate = sample_delta;
+		h->apt_candidate_count = 0;
+		return health_queue;
+	}
+
+	++h->apt_event_count;
+	if (unlikely(h->apt_candidate == sample_delta &&
+		     ++h->apt_candidate_count == c[event_entropy_shift])) {
+		health_apt_reset(h);
+		return health_discard;
+	} else if (c[event_entropy_shift] - h->apt_candidate_count >
+		   n - h->apt_event_count) {
+		/*
+		 * The currently pending APT might not have seen all n
+		 * events yet, but it's already known by now that it
+		 * can't fail anymore. Note that the above condition
+		 * also coverts the case h->apt_event_count == n.
+		 */
+		health_apt_reset(h);
+		return health_dispatch;
+	}
+
+	return health_queue;
+}
+
 static void health_test_reset(struct health_test *h)
-{}
+{
+	health_apt_reset(h);
+}
 
 static enum health_result
 health_test_process(struct health_test *h, unsigned int event_entropy_shift,
@@ -907,7 +959,7 @@ health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 	sample_delta = sample - h->previous_sample;
 	h->previous_sample = sample;
 
-	return health_none;
+	return health_test_apt(h, event_entropy_shift, sample_delta);
 }
 
 struct fast_pool {
-- 
2.26.2

[RFC PATCH 35/41] random: improve the APT's statistical power

[Nicolai Stange]

The Adapative Proportion Test as specified by NIST SP800-90B counts how
often the first sample value in a sequence of n samples occurs among the
remaining n - 1 ones and will report failure if the result is unexpectedly
large. The intention is to capture cases where a noise source's actual
min-entropy falls below the one estimated during the validation process.
Note that, assuming i.i.d., a decrease in per-IRQ min-entropy corresponds
to an increase in the maximum probability among all possible sample values,
per the definition of min-entropy.

For example, consider the maximum supported per-IRQ min-entropy estimate of
H=1, which corresponds to a maximum probability of p = 2^-H = 50% among all
possible sample values. Now, if the actual entropy degraded to H/2, it
would mean that some sample value's likelihood had increased to ~70%. The
ability of the APT to detect this degradation is limited by the way it's
currently implemented: a prerequisite for successfully reporting a
sequence of n samples as bad is to find the offending sample value at the
leading position. Thus, the power of the APT is always limited by the
probability of the offending sample value, i.e. 70% in this example, no
matter how large the total number n of examined of samples is.

This can be improved upon by taking advantage of the fact that only values
of H <= 1 are currently supported for the per-IRQ entropy estimate. It
follows that the maximum probability among all sample values would increase
to > 1/2 in case the actual min-entropy happened to fall below the assumed
value. If we were to examine a sequence of n1 samples, the expected number
of occurrences of the offending sample value would be > 1/2 * n1 (again
assuming i.i.d). For example, for an actual entropy of H/2, with H=1 as
above, the probability to find 4 or more samples of the same value among a
sequence of n1 = 7 events would be ~88%, which is an improvement over the
70% from above.

So partition the total number of samples n = 128/H to examine from the APT
into two parts, n1 and n2, such that n = n1 + n2 with n1 odd. Rather than
simply picking the first sample value to subsequently search for in the
remaining n-1 events, make the APT to run a "presearch" on the first n1
samples in order to find the value occurring more than n1 / 2 times, if
there is such one. Make the APT then continue as usual: let it search the
remaining n2 samples for the found candidate value, count the number of
occurrences and report failure if a certain threshold is reached.

Of course, new thresholds should be installed in order to gain optimal
statistical power from the second phase while still maintaining a false
positive rate of 2^-16 as before. An exhaustive search among all
possibilities for the different choices of n1 and supported per-IRQ
min-entropies revealed that n1 = 7 is optimal for n = 128 (H = 1) and
close to the resp. optimum for larger n, i.e. smaller H. With this choice,
the new presearch scheme yields new thresholds ("c") and probabilities to
detect a entropy degradations to H/2 ("power") as tabulated below:

   H     n   c    power
   --------------------
      1  128   83 64.7%
    1/2  256  205 79.1%
    1/4  512  458 81.6%
    1/8 1024  968 84.0%
   1/16 2048 1991 84.9%
   1/32 4096 4038 86.9%
   1/64 8192 8134 86.4%

Compare this to the former numbers for the original implementation:

   H     n   c    power
   --------------------
      1  128   87 52.5%
    1/2  256  210 67.5%
    1/4  512  463 76.7%
    1/8 1024  973 82.8%
   1/16 2048 1997 82.6%
   1/32 4096 4044 85.8%
   1/64 8192 8140 85.8%

So for smaller values of H, i.e. for H <= 1/8, the improvement isn't really
impressive, but that was to be expected. OTOH, for the larger Hs, that is
for the per-IRQ entropies estimated for systems with a high resolution
get_cycles(), there is a clear advantage over the old scheme.

Implement the described presearch for finding the sample value occurring
more than half of the times among the first n1=7 events in a sequence of
n=128/H samples to examine, if there is such one. Rather than maintaining
individual per-CPU counters for the 2^8 possible sample values each, count
the numbers of ones at the eight resp. bit positions. Note that if some
sample value has indeed been observed more than half of the time, it will
dominate all these bit counters and its value can be unambiguously restored
from them, which is all that is needed.

For better reviewability, represent the eight bit counters as an array of
eight u8's at struct health_test and implement the bit counting as well
as the final candidate extraction in the most naive way. A follow-up patch
will sequeeze the counters into a single u32 and also optimize the bit
counting and candidate extraction performance-wise.

Implement the new health_apt_presearch_update() for updating the presearch
bit counters. Call it from health_test_apt() on the first n1=7 samples.

Implement the new health_apt_presearch_finalize() for restoring the
candidate from the presearch bit counters. Call it from health_test_apt()
once the n1'th event in a sequence has been processed and the presearch
phase is to be concluded.

Make health_test_apt() search for the candidate value as determined by
the presearch phase among the sequence's remaining n2 = n - n1 samples.
Adapt the failure thresholds to the now slightly smaller n2 values.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 58 +++++++++++++++++++++++++++++++++++++------
 1 file changed, 50 insertions(+), 8 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 131302cbc495..75a103f24fea 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -881,8 +881,13 @@ static void discard_queued_entropy(struct entropy_store *r,
 
 struct health_test {
 	unsigned short apt_event_count;
-	unsigned short apt_candidate_count;
-	u8 apt_candidate;
+	union {
+		u8 apt_presearch_bit_counters[8];
+		struct {
+			unsigned short apt_candidate_count;
+			u8 apt_candidate;
+		};
+	};
 
 	u8 previous_sample;
 };
@@ -895,9 +900,44 @@ enum health_result {
 };
 
 /* Adaptive Proportion Test */
+#define HEALTH_APT_PRESEARCH_EVENT_COUNT 7
+
+static void health_apt_presearch_update(struct health_test *h, u8 sample_delta)
+{
+	int i;
+
+	for (i = 0; i < 8; ++i) {
+		h->apt_presearch_bit_counters[i] = sample_delta & 0x1;
+		sample_delta >>= 1;
+	}
+}
+
+static void health_apt_presearch_finalize(struct health_test *h)
+{
+	int i;
+
+	/*
+	 * If some event octet occurred more than half of the time,
+	 * i.e. more than HEALTH_APT_PRESEARCH_EVENT_COUNT / 2 times,
+	 * then its value can be restored unambigiously from the eight
+	 * ->apt_presearch_bit_counters each holding the count of 1s
+	 * encountered at the corresponding bit positions.
+	 */
+	h->apt_candidate = 0;
+	for (i = 0; i < 8; ++i) {
+		if (h->apt_presearch_bit_counters[i] >=
+		    (HEALTH_APT_PRESEARCH_EVENT_COUNT + 1) / 2) {
+			h->apt_candidate |= 1 << i;
+		}
+	}
+	h->apt_candidate_count = 0;
+};
+
 static void health_apt_reset(struct health_test *h)
 {
 	h->apt_event_count = 0;
+	memset(h->apt_presearch_bit_counters, 0,
+		sizeof(h->apt_presearch_bit_counters));
 }
 
 static enum health_result
@@ -911,16 +951,18 @@ health_test_apt(struct health_test *h, unsigned int event_entropy_shift,
 	 * values of event_entropy_shift each, should have probability
 	 * <= 2^-16.
 	 */
-	static const unsigned int c[] = {87, 210, 463, 973, 1997, 4044, 8140};
+	static const unsigned int c[] = {83, 205, 458, 968, 1991, 4038, 8134};
+
+	BUILD_BUG_ON(HEALTH_APT_PRESEARCH_EVENT_COUNT != 7);
 
-	if (!h->apt_event_count) {
-		h->apt_event_count = 1;
-		h->apt_candidate = sample_delta;
-		h->apt_candidate_count = 0;
+	++h->apt_event_count;
+	if (unlikely(h->apt_event_count <= HEALTH_APT_PRESEARCH_EVENT_COUNT)) {
+		health_apt_presearch_update(h, sample_delta);
+		if (h->apt_event_count == HEALTH_APT_PRESEARCH_EVENT_COUNT)
+			health_apt_presearch_finalize(h);
 		return health_queue;
 	}
 
-	++h->apt_event_count;
 	if (unlikely(h->apt_candidate == sample_delta &&
 		     ++h->apt_candidate_count == c[event_entropy_shift])) {
 		health_apt_reset(h);
-- 
2.26.2

[RFC PATCH 36/41] random: optimize the APT's presearch

[Nicolai Stange]

The Adaptive Proportion Test's presearch phase is supposed to determine the
sample value occurring more than half of the times among the first n1=7
events in a sequence, if there is any such one.

To this end, it maintains eight counters at struct health_test for counting
the numbers of ones observed at each of the eight resp. bit positions
within the sample values' octets. The idea is that if any sample value had
been encountered more than half of the time, it would dominate all these
counters and its value could be restored unambiguously from them.

For better reviewability, this had been implemented in the most
straightforward way:
- the counters had been represented as an array of eight u8s at struct
  health_test and
- both, the counter updating as well as the candidate extracion code had
  been implemented by means of a loop over the eight bit positions.

As this is all accessed from add_interrupt_randomness(), optimizations
won't harm. In particular, using a total of eight bytes for the bit
counters is wasteful and can be reduced to half of that, optimizing the
size of struct health_test, which in turn is stored as part of the per-CPU
struct fast_pool.

For counts up to n1=7, 3 bits would suffice. Rather than using an array of
eight u8s, store the bit counter within an u32's eight four-bit nibbles.

Even though it probably won't matter on average in practice, because the
APT presearch is run only for a small fraction of all IRQ events, reduce
the number of instructions issued from the bit counter maintenance and
candidate extraction code as well. If nothing else, this can potentially
reduce the maximum IRQ latency by a few cycles.

Namely, make the bit counter updating code in health_apt_presearch_update()
spread the eight bits from the input sample evenly across an u32. That is,
the bit at position i will end up at position 4*i. This can be achieved
within five binops and four shifts. This intermediate value can then get
subsequently added in a single operation to the "packed" bit counters kept
in struct health_test in order to conclude the operation.

As for the final candidate extraction in health_apt_presearch_finalize(),
remember that the i'th counter needs to get compared against
n1 / 2 = 7 / 2 in order to restore the i'th bit from the resulting
candidate value. The condition that one such bit counter is >= 4 is
equivalent to testing its bit at the 2nd position, counted from zero.
Thus, (->apt_presearch_bit_counters & 0x44444444) >> 2
will yield a value where the LSB from the i'th nibble, equals the i'th bit
from the result and everything else is unset. The final result can then be
obtained by "shrinking" this intermediate representation back into an u8.
In total, the candidate extraction can be achieved within a sequence of
seven binops and six shifts.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 71 ++++++++++++++++++++++++++++++++-----------
 1 file changed, 53 insertions(+), 18 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 75a103f24fea..2c744d2a9b26 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -882,7 +882,7 @@ static void discard_queued_entropy(struct entropy_store *r,
 struct health_test {
 	unsigned short apt_event_count;
 	union {
-		u8 apt_presearch_bit_counters[8];
+		u32 apt_presearch_bit_counters;
 		struct {
 			unsigned short apt_candidate_count;
 			u8 apt_candidate;
@@ -904,40 +904,75 @@ enum health_result {
 
 static void health_apt_presearch_update(struct health_test *h, u8 sample_delta)
 {
-	int i;
+	u32 encoded;
 
-	for (i = 0; i < 8; ++i) {
-		h->apt_presearch_bit_counters[i] = sample_delta & 0x1;
-		sample_delta >>= 1;
-	}
+	/*
+	 * Encode the sample octet by "widening" it into 8
+	 * nibbles. That is, bit i from the source will be assigned to
+	 * bit 4*i in the result. All other bits are set to zero.
+	 */
+	encoded = sample_delta;
+	encoded = (encoded & 0xf) | ((encoded >> 4) << 16);
+	encoded |= (encoded << 6);
+	encoded |= (encoded << 3);
+	encoded &= 0x11111111;
+
+	/*
+	 * The nibbles from ->apt_presearch_bit_counters, each
+	 * counting the number of occurences of 1s at the
+	 * corresponding bit position, don't overflow into each other.
+	 */
+	BUILD_BUG_ON(ilog2(HEALTH_APT_PRESEARCH_EVENT_COUNT) >= 4);
+	h->apt_presearch_bit_counters += encoded;
 }
 
 static void health_apt_presearch_finalize(struct health_test *h)
 {
-	int i;
+	u32 majority, decoded;
 
 	/*
 	 * If some event octet occurred more than half of the time,
 	 * i.e. more than HEALTH_APT_PRESEARCH_EVENT_COUNT / 2 times,
 	 * then its value can be restored unambigiously from the eight
-	 * ->apt_presearch_bit_counters each holding the count of 1s
-	 * encountered at the corresponding bit positions.
+	 * ->apt_presearch_bit_counters nibbles each holding the count
+	 * of 1s encountered at the corresponding bit positions.
+	 *
+	 * Because HEALTH_APT_PRESEARCH_EVENT_COUNT is a power of two
+	 * minus one, the condition
+	 * nibble >=  HEALTH_APT_PRESEARCH_EVENT_COUNT / 2
+	 * is true iff the nibble's bit at postion
+	 * ilog2(HEALTH_APT_PRESEARCH_EVENT_COUNT) is set.
 	 */
-	h->apt_candidate = 0;
-	for (i = 0; i < 8; ++i) {
-		if (h->apt_presearch_bit_counters[i] >=
-		    (HEALTH_APT_PRESEARCH_EVENT_COUNT + 1) / 2) {
-			h->apt_candidate |= 1 << i;
-		}
-	}
+	BUILD_BUG_ON(!is_power_of_2(HEALTH_APT_PRESEARCH_EVENT_COUNT + 1));
+#define MAJORITY_BIT ilog2(HEALTH_APT_PRESEARCH_EVENT_COUNT)
+#define SHL_AND_OR(a, shl) ((a) | ((a) << shl))
+#define SET_ALL_NIBBLES_TO(to) SHL_AND_OR(SHL_AND_OR(SHL_AND_OR(to, 16), 8), 4)
+#define MAJORITY_MASK SET_ALL_NIBBLES_TO(1 << MAJORITY_BIT)
+	majority = (h->apt_presearch_bit_counters & MAJORITY_MASK);
+	majority >>= MAJORITY_BIT;
+#undef MAJORITY_MASK
+#undef SET_ALL_NIBBLES_TO
+#undef MAJORITY_BIT
+
+	/*
+	 * Reverse the encoding from health_apt_update_presearch().
+	 * That is, "shrink" the eight nibbles back into an octet such
+	 * that the result's i'th bit is set to the i'th nibble's LSB.
+	 */
+	decoded = majority;
+	decoded |= (decoded >> 3);
+	decoded |= (decoded >> 3);
+	decoded |= (decoded >> 3);
+	decoded = (((decoded >> 16) << 4) | (decoded & 0xf)) & 0xff;
+
+	h->apt_candidate = decoded;
 	h->apt_candidate_count = 0;
 };
 
 static void health_apt_reset(struct health_test *h)
 {
 	h->apt_event_count = 0;
-	memset(h->apt_presearch_bit_counters, 0,
-		sizeof(h->apt_presearch_bit_counters));
+	h->apt_presearch_bit_counters = 0;
 }
 
 static enum health_result
-- 
2.26.2

[RFC PATCH 37/41] random: implement the "Repetition Count" NIST SP800-90B health test

[Nicolai Stange]

The "Repetition Count Test" (RCT) as specified by NIST SP800-90B simply
counts the number of times the same sample value has been observed and
reports failure if an highly unlikely threshold is exceeded. The exact
values of the latter depend on the estimated per-IRQ min-entropy H as well
as on the upper bounds set on the probability of false positives. For the
latter, a maximum value of 2^-20 is recommended and with this value the
threshold can be calculated as 1 + ceil(20 / H). It should be noted that
the RCT has very poor statistical power and is only intended to detect
catastrophic noise source failures, like the get_cycles() in
add_interrupt_randomness() always returning the same constant.

Add the fields needed for maintaining the RCT state to struct health_test:
->rct_previous_delta for storing the previous sample value and ->rct_count
for keeping track of how many times this value has been observed in a row
so far.

Implement the RCT and wrap it in a new function, health_test_rct().

Make the health test entry point, health_test_process(), call it early
before invoking the APT and forward failure reports to the caller. All
other return codes from the RCT are ignored, because
- as said, the statistical power is weak and a positive outcome wouldn't
  tell anything and
- it's not desirable to make the caller, i.e. add_interrupt_randomness(),
  to further queue any entropy once the concurrently running APT has
  signaled a successful completion.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 55 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 2c744d2a9b26..54ee082ca4a8 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -890,6 +890,9 @@ struct health_test {
 	};
 
 	u8 previous_sample;
+
+	u8 rct_previous_delta;
+	unsigned short rct_count;
 };
 
 enum health_result {
@@ -899,6 +902,43 @@ enum health_result {
 	health_discard,
 };
 
+/* Repetition count test. */
+static enum health_result
+health_test_rct(struct health_test *h, unsigned int event_entropy_shift,
+		u8 sample_delta)
+{
+	unsigned int c;
+
+	if (likely(sample_delta != h->rct_previous_delta)) {
+		h->rct_previous_delta = sample_delta;
+		h->rct_count = 0;
+		return health_dispatch;
+	}
+
+	h->rct_count++;
+	if (!h->rct_count) {
+		/* Overflow. */
+		h->rct_count = -1;
+	}
+
+	/*
+	 * With a min-entropy of H = 2^-event_entropy_shift bits per
+	 * event, the maximum probability of seing any particular
+	 * sample value (i.e. delta) is bounded by 2^-H. Thus, the
+	 * probability to observe the same events C times in a row is
+	 * less than 2^-((C - 1) * H). Limit the false positive rate
+	 * of the repetition count test to 2^-20, which yields a
+	 * cut-off value of C = 1 + 20/H. Note that the actual number
+	 * of repetitions equals ->rct_count + 1, so this offset by
+	 * one must be accounted for in the comparison below.
+	 */
+	c = 20 << event_entropy_shift;
+	if (h->rct_count >= c)
+		return health_discard;
+
+	return health_queue;
+}
+
 /* Adaptive Proportion Test */
 #define HEALTH_APT_PRESEARCH_EVENT_COUNT 7
 
@@ -1027,6 +1067,7 @@ health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 		    u8 sample)
 {
 	u8 sample_delta;
+	enum health_result rct;
 
 	/*
 	 * The min-entropy estimate has been made for the lower eight
@@ -1036,6 +1077,20 @@ health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 	sample_delta = sample - h->previous_sample;
 	h->previous_sample = sample;
 
+	rct = health_test_rct(h, event_entropy_shift, sample_delta);
+	if (rct == health_discard) {
+		/*
+		 * Something is really off, get_cycles() has become
+		 * (or always been) a constant.
+		 */
+		return health_discard;
+	}
+
+	/*
+	 * Otherwise return whatever the APT returns. In particular,
+	 * don't care about whether the RCT needs to consume more
+	 * samples to complete.
+	 */
 	return health_test_apt(h, event_entropy_shift, sample_delta);
 }
 
-- 
2.26.2

[RFC PATCH 38/41] random: enable NIST SP800-90B startup tests

[Nicolai Stange]

NIST SP800-90B, section 4.3 requires an entropy source to inhibit output
until the so-called "startup" tests have completed. These "startup" test
shall process at least 1024 consecutive samples by means of the continuous
health tests, i.e. the already implemented Repetition Count Test (RCT) and
Adaptive Proportion Test (APT).

Introduce a new field ->warmup to struct health_test. Initialize it to 1024
from health_test_reset(). Make health_test_process() decrement ->warmup
once per event processed without test failure, but reset ->warmup to the
intitial value upon failure. Prevent health_test_process() from returning
health_dispatch as long as ->warmup hasn't dropped to zero. This will cause
the caller, i.e. add_interrupt_randomness(), to not dispatch any entropy to
the global balance until the startup tests have finished.

Note that this change will delay the initial seeding of the primary_crng,
especially for those values of the estimated per-IRQ min-entropy H where
the mimimum of 1024 events from above is by several factors larger than
128/H, the number of events to be processed by a single APT run. That
would only affect systems running with fips_enabled though and there's
simply no way to avoid it without violating the specs.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 54ee082ca4a8..bd8c24e433d0 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -880,6 +880,7 @@ static void discard_queued_entropy(struct entropy_store *r,
 }
 
 struct health_test {
+	unsigned short warmup;
 	unsigned short apt_event_count;
 	union {
 		u32 apt_presearch_bit_counters;
@@ -1059,6 +1060,13 @@ health_test_apt(struct health_test *h, unsigned int event_entropy_shift,
 
 static void health_test_reset(struct health_test *h)
 {
+	/*
+	 * Don't dispatch until at least 1024 events have been
+	 * processed by the continuous health tests as required by
+	 * NIST SP800-90B for the startup tests.
+	 */
+	h->warmup = 1024;
+
 	health_apt_reset(h);
 }
 
@@ -1067,7 +1075,7 @@ health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 		    u8 sample)
 {
 	u8 sample_delta;
-	enum health_result rct;
+	enum health_result rct, apt;
 
 	/*
 	 * The min-entropy estimate has been made for the lower eight
@@ -1083,6 +1091,8 @@ health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 		 * Something is really off, get_cycles() has become
 		 * (or always been) a constant.
 		 */
+		if (h->warmup)
+			health_test_reset(h);
 		return health_discard;
 	}
 
@@ -1091,7 +1101,18 @@ health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 	 * don't care about whether the RCT needs to consume more
 	 * samples to complete.
 	 */
-	return health_test_apt(h, event_entropy_shift, sample_delta);
+	apt = health_test_apt(h, event_entropy_shift, sample_delta);
+	if (unlikely(h->warmup) && --h->warmup) {
+		if (apt == health_discard)
+			health_test_reset(h);
+		/*
+		 * Don't allow the caller to dispatch until warmup
+		 * has completed.
+		 */
+		return apt == health_dispatch ? health_queue : apt;
+	}
+
+	return apt;
 }
 
 struct fast_pool {
-- 
2.26.2

[RFC PATCH 39/41] random: make the startup tests include muliple APT invocations

[Nicolai Stange]

Given a per-IRQ min-entropy estimate of H, the Adaptive Proportion Tests
(APT) will need to consume at most n = 128/H samples before reaching a
conclusion. The supported values for H are 1, 1/2, 1/4, 1/8, ..., 1/64,
but only 1 and 1/8 are currently being actively used on systems with and
without a high resolution get_cycles() respectively. The corresponding
number of samples consumed by one APT execution are 128, 256, 512, 1024,
2048, 4096 and 8192. Currently, the ->warmup parameter used for controlling
the startup is hardcoded to be initialized to 1024 and the health test
logic won't permit the caller, i.e. add_interrupt_randomness() to dispatch
any entropy to the global balance until that many events have been
processed *and* the first APT has completed, whichever comes later. It
would take roughly eight successful APT invocations for H=1 until the
startup sequence has completed, but for all H <= 1/8, the ->warmup logic is
effectively useless because the first APT would always need to process
>= 1024 samples anyway.

The probabilites of one single APT invocation successfully detecting a
degradation of the per-IRQ min-entopy to H/2 ("power") are as follows for
the different supported H estimates:

   H     n   power
   ---------------
      1  128 64.7%
    1/2  256 79.1%
    1/4  512 81.6%
    1/8 1024 84.0%
   1/16 2048 84.9%
   1/32 4096 86.9%
   1/64 8192 86.4%

Thus, for H=1, the probability that at least one out of those eight APT
invocations will detect a degradation to H/2 is 1 - (1 - 64.7%)^8 = 99.98%,
which is quite good. OTOH, the 84.0% achievable with the single APT
invocation for H = 1/8 is only semi-satisfactory.

Note that as it currently stands, the only point in time where the health
tests can still intervene and keep back low quality noise from
the primary_crng is before the initial seed has happened. Afterwards,
failing continuous health tests would only potentially delay those best
effort reseeds (which is questionable behaviour in itself, as the crng
state's entropy is never reduced in the course of reseeding).

A future patch will enable dynamically switching from the initial H=1 or
1/8 resp. to lower per-IRQ entropy values upon health test failures in
order to keep those systems going where these more or less arbitrary
per-IRQ entropy estimates turn out to be simply wrong. From a paranoia POV,
it is certainly a good idea to run the APT several times in a row during
startup in order to achieve a good statistical power. Extending the warmup
to cover the larger of the 1024 events required by NIST SP800-90B and four
full APT lengths will result in a combined probability of detecting an
entropy degradation to H/2 of >= 99.98% across all supported values of H.
The obvious downside is that the number of IRQ events required for the
initial seed will be qadrupled, at least for H <= 1/8.

Follow this approach. Amend health_test_reset()'s signature by an
additional parameter, event_entropy_shift, and make it set ->warmup to
the larger of 1024 and 4 * 128 / (2^-event_entropy_shift). Adjust all
call sites accordingly.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 23 +++++++++++++++--------
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index bd8c24e433d0..86dd87588b1b 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1058,14 +1058,21 @@ health_test_apt(struct health_test *h, unsigned int event_entropy_shift,
 	return health_queue;
 }
 
-static void health_test_reset(struct health_test *h)
+static void health_test_reset(struct health_test *h,
+			      unsigned int event_entropy_shift)
 {
 	/*
-	 * Don't dispatch until at least 1024 events have been
-	 * processed by the continuous health tests as required by
-	 * NIST SP800-90B for the startup tests.
+	 * Let H = 2^-event_entropy_shift equal the estimated per-IRQ
+	 * min-entropy. One APT will consume at most 128 / H samples
+	 * until completion. Run the startup tests for the larger of
+	 * 1024 events as required by NIST or four times the APT
+	 * length. In either case, the combined probability of the
+	 * resulting number of successive APTs to detect a degradation
+	 * of H to H/2 will be >= 99.8%, for any supported value of
+	 * event_entropy_shift.
 	 */
-	h->warmup = 1024;
+	h->warmup = 4 * (128 << event_entropy_shift);
+	h->warmup = max_t(unsigned int, h->warmup, 1024);
 
 	health_apt_reset(h);
 }
@@ -1092,7 +1099,7 @@ health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 		 * (or always been) a constant.
 		 */
 		if (h->warmup)
-			health_test_reset(h);
+			health_test_reset(h, event_entropy_shift);
 		return health_discard;
 	}
 
@@ -1104,7 +1111,7 @@ health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 	apt = health_test_apt(h, event_entropy_shift, sample_delta);
 	if (unlikely(h->warmup) && --h->warmup) {
 		if (apt == health_discard)
-			health_test_reset(h);
+			health_test_reset(h, event_entropy_shift);
 		/*
 		 * Don't allow the caller to dispatch until warmup
 		 * has completed.
@@ -1883,7 +1890,7 @@ static inline void fast_pool_init_accounting(struct fast_pool *f)
 		return;
 
 	f->event_entropy_shift = min_irq_event_entropy_shift();
-	health_test_reset(&f->health);
+	health_test_reset(&f->health, f->event_entropy_shift);
 }
 
 void add_interrupt_randomness(int irq, int irq_flags)
-- 
2.26.2

[RFC PATCH 40/41] random: trigger startup health test on any failure of the health tests

[Nicolai Stange]

The startup health tests to be executed at boot as required by NIST 800-90B
consist of running the contiuous health tests, i.e. the Adaptive Proportion
Test (APT) and the Repetition Count Test (RCT), until a certain amount
of noise samples have been examined. In case of test failure during this
period, the startup tests would get restarted by means of reinitializing
the fast_pool's ->warmup member with the original number of total samples
to examine during startup.

A future patch will enable dynamically switching from the initial H=1 or
1/8 per-IRQ min-entropy estimates to lower values upon health test
failures in order to keep those systems going where these more or less
arbitrary per-IRQ entropy estimates turn out to simply be wrong. It is
certainly desirable to restart the startup health tests upon such a switch.

In order to keep the upcoming code comprehensible, move the startup test
restart logic from health_test_process() into add_interrupt_randomness().
For simplicity, make add_interrupt_randomness() trigger a startup test on
each health test failure. Note that there's a change in behaviour: up to
now, only the bootime startup tests would have restarted themselves upon
failure, whereas now even a failure of the continuous health tests can
potentially trigger a startup test long after boot.

Note that as it currently stands, rerunning the full startup tests after
the crng has received its initial seed has the only effect to inhibit
entropy dispatch for a while and thus, to potentially delay those best
effort crng reseeds during runtime. As reseeds never reduce a crng state's
entropy, this behaviour is admittedly questionable. However, further
patches introducing forced reseeds might perhaps become necessary in the
future, c.f. the specification of "reseed_interval" in NIST SP800-90A.
Thus, it's better to keep the startup health test restart logic consistent
for now.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index 86dd87588b1b..bb79dcb96882 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1098,8 +1098,6 @@ health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 		 * Something is really off, get_cycles() has become
 		 * (or always been) a constant.
 		 */
-		if (h->warmup)
-			health_test_reset(h, event_entropy_shift);
 		return health_discard;
 	}
 
@@ -1110,8 +1108,6 @@ health_test_process(struct health_test *h, unsigned int event_entropy_shift,
 	 */
 	apt = health_test_apt(h, event_entropy_shift, sample_delta);
 	if (unlikely(h->warmup) && --h->warmup) {
-		if (apt == health_discard)
-			health_test_reset(h, event_entropy_shift);
 		/*
 		 * Don't allow the caller to dispatch until warmup
 		 * has completed.
@@ -1928,6 +1924,14 @@ void add_interrupt_randomness(int irq, int irq_flags)
 			health_test_process(&fast_pool->health,
 					    fast_pool->event_entropy_shift,
 					    cycles);
+		if (unlikely(health_result == health_discard)) {
+			/*
+			 * Oops, something's odd. Restart the startup
+			 * tests.
+			 */
+			health_test_reset(&fast_pool->health,
+					  fast_pool->event_entropy_shift);
+		}
 	}
 
 	if (unlikely(crng_init == 0)) {
-- 
2.26.2

[RFC PATCH 41/41] random: lower per-IRQ entropy estimate upon health test failure

[Nicolai Stange]

Currently, if fips_enabled is set, a per-IRQ min-entropy estimate of
either 1 bit or 1/8 bit is assumed, depending on whether a high resolution
get_cycles() is available or not. The statistical NIST SP800-90B startup
health tests are run on a certain amount of noise samples and are intended
to reject in case this hypothesis turns out to be wrong, i.e. if the
actual min-entropy is smaller. As long as the startup tests haven't
finished, entropy dispatch and thus, the initial crng seeding, is
inhibited. On test failure, the startup tests would restart themselves
from the beginning.

It follows that in case a system's actual per-IRQ min-entropy is smaller
than the more or less arbitrarily assessed 1 bit or 1/8 bit resp., there
will be a good chance that the initial crng seed will never complete.
AFAICT, such a situation could potentially prevent certain userspace
daemons like OpenSSH from loading.

In order to still be able to make any progress, make
add_interrupt_randomness() lower the per-IRQ min-entropy by one half upon
each health test failure, but only until the minimum supported value of
1/64 bits has been reached. Note that health test failures will cause a
restart of the startup health tests already and thus, a certain number of
additional noise samples resp. IRQ events will have to get examined by the
health tests before the initial crng seeding can take place. This number
of fresh events required is reciprocal to the estimated per-IRQ
min-entropy H: for the Adaptive Proportion Test (APT) it equals ~128 / H.
It follows that this patch won't be of much help for embedded systems or
VMs with poor IRQ rates at boot time, at least not without manual
intervention. But there aren't many options left when fips_enabled is set.

With respect to NIST SP800-90B conformance, this patch enters kind of a
gray area: NIST SP800-90B has no notion of such a dynamically adjusted
min-entropy estimate. Instead, it is assumed that some fixed value has been
estimated based on general principles and subsequently validated in the
course of the certification process. However, I would argue that if a
system had successfully passed certification for 1 bit or 1/8 bit resp. of
estimated min-entropy per sample, it would automatically be approved for
all smaller values as well. Had we started out with such a lower value
passing the health tests from the beginning, the latter would never have
complained in the first place and the system would have come up just fine.

Finally, note that all statistical tests have a non-zero probability of
false positives and so do the NIST SP800-90B health tests. In order to not
keep the estimated per-IRQ entropy at a smaller level than necessary for
forever after spurious health test failures, make
add_interrupt_randomness() attempt to double it again after a certain
number of successful health test passes at the degraded entropy level have
been completed. This threshold should not be too small in order to avoid
excessive entropy accounting loss due to continuously alternating between
a too large per-IRQ entropy estimate and the next smaller value. For now,
choose a value of five as a compromise between quick recovery and limiting
said accounting loss.

So, introduce a new member ->good_tests to struct fast_pool for keeping
track of the number of successfult health test passes. Make
add_interrupt_randomness() increment it upon successful healh test
completion and reset it to zero on failures. Make
add_interrupt_randomness() double the current min-entropy estimate and
restart the startup health in case ->good_tests is > 4 and the entropy
had previously been lowered.

Signed-off-by: Nicolai Stange <nstange@suse.de>
---
 drivers/char/random.c | 25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index bb79dcb96882..24c09ba9d7d0 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1126,6 +1126,7 @@ struct fast_pool {
 	bool		dispatch_needed : 1;
 	bool		discard_needed : 1;
 	int		event_entropy_shift;
+	unsigned int		good_tests;
 	struct queued_entropy	q;
 	struct health_test	health;
 };
@@ -1926,9 +1927,13 @@ void add_interrupt_randomness(int irq, int irq_flags)
 					    cycles);
 		if (unlikely(health_result == health_discard)) {
 			/*
-			 * Oops, something's odd. Restart the startup
-			 * tests.
+			 * Oops, something's odd. Lower the entropy
+			 * estimate and restart the startup tests.
 			 */
+			fast_pool->event_entropy_shift =
+				min_t(unsigned int,
+				      fast_pool->event_entropy_shift + 1, 6);
+			fast_pool->good_tests = 0;
 			health_test_reset(&fast_pool->health,
 					  fast_pool->event_entropy_shift);
 		}
@@ -1951,6 +1956,7 @@ void add_interrupt_randomness(int irq, int irq_flags)
 		 * entropy discard request?
 		 */
 		fast_pool->dispatch_needed = !fast_pool->discard_needed;
+		fast_pool->good_tests++;
 		break;
 
 	case health_discard:
@@ -2005,6 +2011,21 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	if (fast_pool->dispatch_needed || health_result == health_none) {
 		reseed = __dispatch_queued_entropy_fast(r, q);
 		fast_pool->dispatch_needed = false;
+
+		/*
+		 * In case the estimated per-IRQ min-entropy had to be
+		 * lowered due to health test failure, but the lower
+		 * value has proven to withstand the tests for some
+		 * time now, try to give the next better value another
+		 * shot.
+		 */
+		if (unlikely((fast_pool->event_entropy_shift >
+			      min_irq_event_entropy_shift())) &&
+		    fast_pool->good_tests > 4) {
+			fast_pool->event_entropy_shift--;
+			health_test_reset(&fast_pool->health,
+					  fast_pool->event_entropy_shift);
+		}
 	} else if (fast_pool->discard_needed) {
 		int dummy;
 
-- 
2.26.2

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Jason A. Donenfeld]

I haven't looked into the details of this patchset yet, but your
description here indicates to me that this is motivated by FIPS
certification desires, which...worries me. I would like to rewrite the
RNG at some point, and I've started to work on a bunch of designs for
this (and proving them correct, too), but going about this via FIPS
certification or trying to implement some NIST specs is most certainly
the wrong way to go about this, will lock us into subpar crypto for
years, and is basically a waste of time.

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Stephan Mueller]

Am Montag, 21. September 2020, 09:58:16 CEST schrieb Nicolai Stange:

Hi Nicolai,

> Hi all,
> 
> first of all, my apologies for the patch bomb following up in reply to this
> mail here -- it's not meant to receive any serious review at all, but only
> to support the discussion I'm hoping to get going.

Thank you for this effort!
> 
> As some of you might already be aware of, all new submissions for FIPS
> certification will be required to comply with NIST SP800-90B from Nov 7th
> on ([1], sec. 7.18 "Entropy Estimation and Compliance with SP 800-90B").
> For reference: broadly speaking, NIST SP800-90B is about noise sources,
> SP800-90A about the DRBG algorithms stacked on top and SP800-90C about how
> everything is supposed to be glued together. The main requirements from
> SP800-90B are
> - no correlations between different noise sources,
> - to continuously run certain health tests on a noise source's output and
> - to provide an interface enabling access to the raw noise samples for
>   validation purposes.
> 
> To my knowledge, all SP800-90B compliant noise sources available on Linux
> today are either based on the Jitter RNG one way or another or on
> architectural RNGs like e.g. x86's RDSEED or arm64's RNDRRS. Currently,
> there's an in-kernel Jitter RNG implementation getting registered (c.f.
> crypto/drbg.c, (*)) with the Crypto RNG API, which is also accessible from
> userspace via AF_ALG. The userspace haveged ([2]) or jitterentropy
> integrations ([3]) are worth mentioning in this context, too. So in
> summary, I think that for the in-kernel entropy consumers falling under the
> scope of FIPS, the currently only way to stay compliant would be to draw it
> from said Crypto API RNG. For userspace applications there's the additional
> option to invoke haveged and alike.
> 
> OTOH, CPU jitter based techniques are not uncontroversial ([4]). In any
> case, it would certainly be a good idea to mix (xor or whatever) any jitter
> output with entropy obtained from /dev/random (**). If I'm not mistaken,
> the mentioned Crypto API RNG implementation (crypto/drbg.c) follows exactly
> this approach, but doesn't enforce it yet: there's no
> wait_for_random_bytes() and early DRBG invocations could in principle run
> on seeds dominated entirely by jitterentropy. However, this can probably
> get sorted quite easily and thus, one reasonable way towards maintaining
> FIPS resp. SP800-90 compliance would be to
> - make crypto/drbg.c invoke wait_for_random_bytes(),
> - make all relevant in-kernel consumers to draw their random numbers from
>   the Crypto RNG API, if not already the case and
> - convert all relevant userspace to use a SP800-90B conforming Jitter RNG
>   style noise source for compliance reasons, either by invoking the
>   kernel's Crypto RNG API or by diffent means, and mix that with
>   /dev/random.
> 
> Even though this would probably be feasible, I'm not sure that giving up on
> /dev/random being the primary, well established source of randomness in
> favor of each and every userspace crypto library rolling its own entropy
> collection scheme is necessarily the best solution (it might very well be
> though).
> 
> An obvious alternative would be to make /dev/random conform to SP800-90B.
> Stephan Müller posted his "LRNG" patchset ([5]), in which he proposed to
> introduce a second, independent implementation aiming at SP800-90[A-C]
> conformance. However, it's in the 35th iteration now and my impression is
> that there's hardly any discussion happening around this for quite a while
> now. I haven't followed the earlier development, but I can imagine several
> reasons for that:
> - people are not really interested in FIPS or even questioning the whole
>   concept in the first place (c.f. Theodore Ts'o remarks on this topic
>   at [6]),
> - potential reviewers got merely discouraged by the diffstat or

Maybe I followed the Linux principle a bit to much here? Release early, 
release often.

But with the v35, all goals I tried to achieve are now in (namely the last was 
to get rid of any non-cryptographic conditioning functions) and to have a very 
clean data processing / entropy analysis. I do not expect big changes any 
more.

> - people dislike the approach of having two competing implementations for
>   what is basically the same functionality in the kernel.

Is this really so bad considering the security implications on this topic? We 
also have multiple file systems, multiple memory allocators, etc...
> 
> In either case, I figured it might perhaps help further discussion to
> provide at least a rough idea of how bad the existing /dev/random
> implementation would get cluttered when worked towards SP800-90B
> compliance. So I implemented the required health tests for the interrupt
> noise source -- the resulting patches can be found in reply to this mail.
> I'd like to stress(!) that this should really only be considered a first
> step and that there would still be a long way towards a complete solution;
> known open items are listed below. Also, I'm fully aware that making those
> continuous health tests block the best effort primary_crng reseeds upon
> failure is a ridiculous thing to do -- that's again meant for demonstration
> purposes only, c.f. the commit log from the next to last patch. Anyway,
> those of you who are interested in some more details beyond the mere
> diffstat can find them after the list of references below.
> 
> In summary, I can imagine three feasible ways towards SP800-90 compliance:
> 1.) Put the burden on consumers. For in-kernel users this would mean
>     conversion to the Jitter backed Crypto RNG API, in case that hasn't
>     happened yet. Userspace is free to use any approved Jitter based
>     mechanism for compliance reasons, but is encouraged to mix that with
>     /dev/random.
> 2.) Merge Stephan's LRNG. Users/distros would have to decide between either
>     of the two competing implementations at kernel config time.
> 3.) Develop the existing /dev/random towards compliance, ideally w/o
>     affecting !fips_enabled users too much. This would likely require some
>     redundancies as well as some atrocities imposed by the specs.
> 
> I'm looking forward to hearing your opinions and suggestions! In case you
> happen to know of anybody who's not on CC but might potentially be
> interested in FIPS, I'd highly appreciate it if you could point him/her to
> this thread. The usual suspects are probably (enterprise?) distro folks,
> but there might be others I haven't thought of.
> 
> Many thanks for your time!
> 
> Nicolai
> 
> 
> (*) That's an oversimplification for the sake of brevity: actually
>     SP800-90A DRBGs stacked on top of the SP800-90B conforming
>     jitterentropy source get registered with the Crypto API.
> (**) "/dev/random" is used as a synonym for everything related to
>      drivers/char/random.c throughout this mail.
> 
> [1]
> https://csrc.nist.gov/csrc/media/projects/cryptographic-module-validation-p
> rogram/documents/fips140-2/fips1402ig.pdf [2]
> http://www.issihosts.com/haveged/
> [3] http://www.chronox.de/jent/doc/CPU-Jitter-NPTRNG.html
>     c.f. appendices C-E
> [4] https://lwn.net/Articles/642166/
> [5] https://lkml.kernel.org/r/5667034.lOV4Wx5bFT@positron.chronox.de
> [6] https://lkml.kernel.org/r/20170919133959.5fgtioyonlsdyjf5@thunk.org
>     https://lkml.kernel.org/r/20170920011642.cczekznqebf2zq5u@thunk.org
> [7] https://lkml.kernel.org/r/aef70b42-763f-0697-f12e-1b8b1be13b07@gmail.com
> 
> 
> As promised above, some more details on the RFC series sent alongside
> follow. The primary goal was to implement that health test functionality as
> required by SP800-90B for the existing drivers/char/random.c without
> affecting !fips_enabled users in any way. As outlined below, I failed quite
> miserably as far as performance is concerned, but that shouldn't be
> something which cannot get rectified. Kernel version v5.9-rc4 had been used
> as a basis. The series can be logically subdivided into the following
> parts:
> - [1-5]: Preparatory cleanup.
> - [6-17]: Implement support for deferring entropy credit dispatch to the
>   global balance to long after the corresponding pool mixing operation has
>   taken place. Needed for "holding back" entropy until the health tests
>   have finished on the latest pending batch of samples.
> - [18-21]: Move arch_get_random_{seed_,}long() out of the interrupt path.
>   Needed to adhere to how SP800-90C expects multiple noise source to get
>   combined, but is also worthwhile on its own from a performance POV.
> - [22-23]: Don't award entropy to non-SP800-90B conforming architectural
>   RNGs if fips_enabled is set.
> - [24]: Move rand_initialize() to after time_init(). A "fix" for what is
>   currently a non-issue, but it's a prerequisite for the subsequent patch.
> - [25]: Detect cycle counter resolution, subsequently needed for making a
>   per-IRQ entropy assessment.
> - [26-28]: Follow Stephan's LRNG approach in how much entropy gets
>   awarded to what: a lot more than before to add_interrupt_randomness(),
>   none to add_{disk,input}_randomness() anymore.
> - [29-33]: Introduce empty health test stubs and wire them up to
>   add_interrupt_randomness().
> - [34-36]: Implement the Adaptive Proportion Test (APT) as specified by
>   SP800-90B and squeeze some more statistical power out of it.
> - [37]: Implement SP800-90B's Repetition Count Test (RCT).
> - [38-40]: Implement the startup tests, which are nothing but the
>   continuous tests (APT + RCT) run on a specified amount of samples at
>   boot time.
> - [41]: Attempt to keep the system going in case the entropy estimate
>   had been too optimistic and the health tests keep failing.
> 
> As the health tests are run from interrupt context on each sample, a
> performance measurement is due. To this end, I configured a Raspberry Pi 2B
> (ARMv7 Cortex A7) to disable all peripherals, gated a
> 19.2 MHz / 2048 ~= 9.3 kHz clock signal to some edge triggered GPIO and
> function_graph traced add_interrupt_randomness() for 10 min from a busybox
> initramfs. Unfortunately, the results had been a bit disappointing: with
> fips_enabled being unset there had been a runtime degradation of ~12.5% w/o
> SMP and ~5% w/ SMP resp. on average merely due to the application of the
> patches onto the v5.9-rc4 base. However, as the amount of work should not
> have changed much and given that struct fast_pool still fits into a single
> cacheline, I'm optimistic that this can get rectified by e.g. introducing
> a static_key for fips_enabled and perhaps shuffling branches a bit such
> that the !fips_enabled code becomes more linear. OTOH, the impact of
> enabling the health tests by means of setting fips_enabled had not been so
> dramatic: the observed increase in average add_interrupt_randomness()
> runtimes had been 6% w/o SMP and 5% w/ SMP respectively.
> 
> Apart from those well controlled experiments on a RPi, I also did some
> lax benchmarking on my x86 desktop (which has some Intel i9, IIRC).
> More specifically, I simply didn't touch the system and ftraced
> add_interrupt_randomness() for 15 mins. The number of captured events had
> been about 2000 in each configuration. Here the add_interrupt_randomness()
> performance improved greatly: from 4.3 us on average w/o the patches down
> to 2.0 us with the patches applied and fips_enabled. However, I suppose
> this gain was due to the removal of RDSEED from add_interrupt_randomness().
> Indeed, when inspecting the distribution of add_interrupt_randomness()
> runtimes on plain v5.9-rc4 more closely, it can be seen that there's a
> good portion of events (about 1/4th) where add_interrupt_randomness() took
> about 10us. So I think that this comparison isn't really a fair one...
> 
> 
> To the best of my knowledge, these are the remaining open questions/items
> towards full SP800-90[A-C] compliance:
> - There's no (debugfs?) interface for accessing raw samples for validation
>   purposes yet. That would be doable though.

I use the patch 12/13 from my patch series successfully for random.c too.

> - try_to_generate_entropy() should probably get wired up to the health
>   tests as well. More or less straightfoward to implement, too.

Correct, but when assessing this function as part of [1], I found hardly any 
entropy being provided with this component.

[1] https://www.bsi.bund.de/SharedDocs/Downloads/EN/BSI/Publications/Studies/
LinuxRNG/LinuxRNG_EN_V4_1.pdf?__blob=publicationFile&v=2

> - Diverting fast_pool contents into net_rand_state is not allowed (for a
>   related discussion on this topic see [7]).

Totally agreed, this is a bad design.

> - I've been told that SP800-90A is not a hard requirement yet, but I
>   suppose it will eventually become one. This would mean that the chacha20
>   RNG would have to get replaced by something approved for fips_enabled.
> - The sequence of fast_pool -> input_pool -> extract_buf() operations
>   is to be considered a "non-vetted conditioning component" in SP800-90B
>   speak. It would follow that the output can't be estimated as having full
>   entropy, but only 0.999 of its length at max. (c.f. sec. 3.1.5.2). This
>   could be resolved by running a SP800-90A derivation function at CRNG
>   reseeding for fips_enabled. extract_buf(), which is already SHA1 based,
>   could perhaps be transformed into such one as well.

The core issue may very well be the SHA-1: using SHA-1 and folding its output 
in half may be an artificial limit of the upper bound of entropy.

> - The only mention of combining different noise sources I was able to find
>   had been in SP800-90C, sec. 5.3.4 ("Using Multiple Entropy Sources"):
>   it clearly states that the outputs have to be combined by concatenation.
>   add_hwgenerator_randomness() mixes into the same input_pool as
>   add_interrupt_randomness() though and I would expect that this isn't
>   allowed, independent of whether the noise source backing the former
>   is SP800-90B compliant or not. IIUC, Stephan solved this for his LRNG
>   by maintaing a separate pool for the hw generator.

See 90B section 3.1.6: combining both via vetted conditioning function is 
accepted. 

> - SP800-90A sets an upper bound on how many bits may be drawn from a
>   DRBG/crng before a reseed *must* take place ("reseed_interval"). In
>   principle that shouldn't matter much in practice, at least not with
>   CONFIG_NUMA: with reseed_interval == 2^32 bits, a single CRNG instance
>   would be allowed to hand out only 500MB worth of randomness before
>   reseeding, but a (single) numa crng chained to the primary_crng may
>   produce as much as 8PB before the latter must eventually get reseeded
>   from the input_pool. But AFAICT, a SP800-90A conforming implementation
>   would still have to provide provisions for a blocking extract_crng().

The hard limit is 2^48 generate operation with up to 2^16 bytes each. This is 
a very large amount of data. If we start triggering reseeds early enough, a 
hard blocking may not be needed.

> - It's entirely unclear to me whether support for "prediction resistance
>   requests" is optional. It would be a pity if it weren't, because IIUC
>   that would effectively imply a return to the former blocking_pool
>   behaviour, which is obviously a no-no.

This is not required.


Ciao
Stephan

Re: [RFC PATCH 22/41] random: introduce arch_has_sp800_90b_random_seed()

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 4213 bytes --]

Hi Nicolai,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on char-misc/char-misc-testing]
[also build test ERROR on arm64/for-next/core powerpc/next s390/features tip/x86/core linus/master v5.9-rc6 next-20200921]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Nicolai-Stange/random-possible-ways-towards-NIST-SP800-90B-compliance/20200921-160315
base:   https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git 8fd0e2a6df262539eaa28b0a2364cca10d1dc662
config: arm64-allyesconfig (attached as .config)
compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from include/linux/random.h:121,
                    from arch/arm64/include/asm/pointer_auth.h:6,
                    from arch/arm64/include/asm/processor.h:39,
                    from include/linux/mutex.h:19,
                    from include/linux/kernfs.h:12,
                    from include/linux/sysfs.h:16,
                    from include/linux/kobject.h:20,
                    from include/linux/of.h:17,
                    from include/linux/irqdomain.h:35,
                    from include/linux/acpi.h:13,
                    from include/acpi/apei.h:9,
                    from include/acpi/ghes.h:5,
                    from include/linux/arm_sdei.h:8,
                    from arch/arm64/kernel/asm-offsets.c:10:
>> arch/arm64/include/asm/archrandom.h:45:8: error: unknown type name 'inine'
      45 | static inine bool arch_has_sp800_90b_random_seed(void)
         |        ^~~~~
>> arch/arm64/include/asm/archrandom.h:45:19: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'arch_has_sp800_90b_random_seed'
      45 | static inine bool arch_has_sp800_90b_random_seed(void)
         |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   make[2]: *** [scripts/Makefile.build:117: arch/arm64/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1198: prepare0] Error 2
   make[1]: Target 'prepare' not remade because of errors.
   make: *** [Makefile:185: __sub-make] Error 2
   make: Target 'prepare' not remade because of errors.

# https://github.com/0day-ci/linux/commit/af235fdbda320942f96e2530990620871f47c9d0
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Nicolai-Stange/random-possible-ways-towards-NIST-SP800-90B-compliance/20200921-160315
git checkout af235fdbda320942f96e2530990620871f47c9d0
vim +/inine +45 arch/arm64/include/asm/archrandom.h

    34	
    35	static inline bool arch_has_random_seed(void)
    36	{
    37		/*
    38		 * Only support the generic interface after we have detected
    39		 * the system wide capability, avoiding complexity with the
    40		 * cpufeature code and with potential scheduling between CPUs
    41		 * with and without the feature.
    42		 */
    43		return cpus_have_const_cap(ARM64_HAS_RNG);
    44	}
  > 45	static inine bool arch_has_sp800_90b_random_seed(void)
    46	{
    47		/*
    48		 * Section K.12.1 from the Arm Architecture Reference Manual
    49		 * Armv8" (DDI0487F) sounds like RNDRRS could qualify as a
    50		 * NIST SP800-90C NRBG, but we're currently using RNDR only.
    51		 */
    52		return false;
    53	}
    54	static inline bool __must_check arch_get_random_long(unsigned long *v)
    55	{
    56		return false;
    57	}
    58	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 74130 bytes --]

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Torsten Duwe]

On Mon, Sep 21, 2020 at 10:40:37AM +0200, Stephan Mueller wrote:
> Am Montag, 21. September 2020, 09:58:16 CEST schrieb Nicolai Stange:
> 
> > - people dislike the approach of having two competing implementations for
> >   what is basically the same functionality in the kernel.
> 
> Is this really so bad considering the security implications on this topic? We 
> also have multiple file systems, multiple memory allocators, etc...

Exactly. I thought Linux was about the freedom of choice. Some people choose
to get a FIPS certification for their Linux-based products, which mostly
means to restrict crypto capabilities to an "allowed" set, granted. But in
this case people might opt for some sort of "entropy QA". I find it hard to
accept that this option is suppressed, especially if it's because of personal
antipathy of the maintainer about the origin of this change and not for
technical reasons. Restrictions on cryptographic functionality are ok, but
health tests on entropy sources are not?

I do understand people's reluctance after the dual-ECC DRBG desaster, but
OTOH SElinux is generally considered an improvement. Definitely not
everything coming from that direction is tainted.

A big portion of this patch set is cleanup, another one said introduction of
entropy source monitoring. This is important, no matter what your attitude
towards certifications might be.

	Torsten

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Greg Kroah-Hartman]

On Tue, Sep 22, 2020 at 03:23:44PM +0200, Torsten Duwe wrote:
> On Mon, Sep 21, 2020 at 10:40:37AM +0200, Stephan Mueller wrote:
> > Am Montag, 21. September 2020, 09:58:16 CEST schrieb Nicolai Stange:
> > 
> > > - people dislike the approach of having two competing implementations for
> > >   what is basically the same functionality in the kernel.
> > 
> > Is this really so bad considering the security implications on this topic? We 
> > also have multiple file systems, multiple memory allocators, etc...
> 
> Exactly. I thought Linux was about the freedom of choice.

http://www.islinuxaboutchoice.com/

:)

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Torsten Duwe]

On Tue, 22 Sep 2020 18:21:52 +0200
Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:

> On Tue, Sep 22, 2020 at 03:23:44PM +0200, Torsten Duwe wrote:
> > On Mon, Sep 21, 2020 at 10:40:37AM +0200, Stephan Mueller wrote:
> > > Am Montag, 21. September 2020, 09:58:16 CEST schrieb Nicolai
> > > Stange:
> > > 
> > > > - people dislike the approach of having two competing
> > > > implementations for what is basically the same functionality in
> > > > the kernel.
> > > 
> > > Is this really so bad considering the security implications on
> > > this topic? We also have multiple file systems, multiple memory
> > > allocators, etc...
> > 
> > Exactly. I thought Linux was about the freedom of choice.
> 
> http://www.islinuxaboutchoice.com/
> 
> :)

Talk is cheap.

gzip -dc /proc/config.gz | wc -l
9789

:-P
	Torsten

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Torsten Duwe]

Almost two weeks passed and these are the "relevant" replies:

Jason personally does not like FIPS, and is afraid of
"subpar crypto". Albeit this patch set strictly isn't about
crypto at all; the crypto subsystem is in the unlucky position
to just depend on a good entropy source.

Greg claims that Linux (kernel) isn't about choice, which is clearly
wrong.

And this is all ???

There are options for stack protection. I can see bounds checking
and other sanity checks all over the place. And doing a similar thing
on entropy sources is a problem?

Admittedly, if entropy sources fail, the kernel will happily remain
running. No bad immediate effects in userland will arise. Only some
cryptographic algorithms, otherwise very decent, will run on
unneccessarily weak keys, probably causing some real-world problems.
Does anybody care?
The NIST and the BSI do, but that does not mean their solutions are
automatically wrong or backdoored.

There is now a well layed-out scheme to ensure quality randomness,
and a lot of work here has been put into its implementation.

Would some maintainer please comment on potential problems or
shortcomings? Otherwise a "Thanks, applied" would be appropriate, IMO.

	Torsten

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Willy Tarreau]

On Fri, Oct 02, 2020 at 02:38:36PM +0200, Torsten Duwe wrote:
> Almost two weeks passed and these are the "relevant" replies:
> 
> Jason personally does not like FIPS, and is afraid of
> "subpar crypto". Albeit this patch set strictly isn't about
> crypto at all; the crypto subsystem is in the unlucky position
> to just depend on a good entropy source.
> 
> Greg claims that Linux (kernel) isn't about choice, which is clearly
> wrong.

I think there's a small misunderstanding here, my understanding is
that for quite a while, the possibilities offered by the various
random subsystems or their proposed derivative used to range from
"you have to choose between a fast system that may be vulnerable
to some attacks, a system that might not be vulnerable to certain
attacks but might not always boot, or a slow system not vulnerable
to certain attacks". Greg's point seems to be that if we add an
option, it means it's yet another tradeoff between these possibilities
and that someone will still not be happy at the end of the chain. If
the proposed solution covers everything at once (performance,
reliability, unpredictability), then there probably is no more reason
for keeping alternate solutions at all, hence there's no need to give
the user the choice between multiple options when only one is known
to always be valid. At least that's how I see it and it makes sense
to me.

> And this is all ???

Possibly a lot of people got used to seeing the numerous versions
and are less attentive to new series, it's possible that your message
will wake everyone up.

Regards,
Willy

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Greg Kroah-Hartman]

On Fri, Oct 02, 2020 at 03:15:55PM +0200, Willy Tarreau wrote:
> On Fri, Oct 02, 2020 at 02:38:36PM +0200, Torsten Duwe wrote:
> > Almost two weeks passed and these are the "relevant" replies:
> > 
> > Jason personally does not like FIPS, and is afraid of
> > "subpar crypto". Albeit this patch set strictly isn't about
> > crypto at all; the crypto subsystem is in the unlucky position
> > to just depend on a good entropy source.
> > 
> > Greg claims that Linux (kernel) isn't about choice, which is clearly
> > wrong.
> 
> I think there's a small misunderstanding here, my understanding is
> that for quite a while, the possibilities offered by the various
> random subsystems or their proposed derivative used to range from
> "you have to choose between a fast system that may be vulnerable
> to some attacks, a system that might not be vulnerable to certain
> attacks but might not always boot, or a slow system not vulnerable
> to certain attacks". Greg's point seems to be that if we add an
> option, it means it's yet another tradeoff between these possibilities
> and that someone will still not be happy at the end of the chain. If
> the proposed solution covers everything at once (performance,
> reliability, unpredictability), then there probably is no more reason
> for keeping alternate solutions at all, hence there's no need to give
> the user the choice between multiple options when only one is known
> to always be valid. At least that's how I see it and it makes sense
> to me.

Thanks for spelling it out in much more detail than I was willing to :)

thanks,

greg k-h

RE: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Van Leeuwen, Pascal]

Torsten,

Ok, if you must have more replies then I'll bite :-)

> -----Original Message-----
> From: Torsten Duwe <duwe@lst.de>
> Sent: Friday, October 2, 2020 2:39 PM
> To: Theodore Y. Ts'o <tytso@mit.edu>
> Cc: linux-crypto@vger.kernel.org; Nicolai Stange <nstange@suse.de>; LKML <linux-kernel@vger.kernel.org>; Arnd Bergmann
> <arnd@arndb.de>; Greg Kroah-Hartman <gregkh@linuxfoundation.org>; Eric W. Biederman <ebiederm@xmission.com>; Alexander
> E. Patrakov <patrakov@gmail.com>; Ahmed S. Darwish <darwish.07@gmail.com>; Willy Tarreau <w@1wt.eu>; Matthew Garrett
> <mjg59@srcf.ucam.org>; Vito Caputo <vcaputo@pengaru.com>; Andreas Dilger <adilger.kernel@dilger.ca>; Jan Kara <jack@suse.cz>;
> Ray Strode <rstrode@redhat.com>; William Jon McCann <mccann@jhu.edu>; zhangjs <zachary@baishancloud.com>; Andy Lutomirski
> <luto@kernel.org>; Florian Weimer <fweimer@redhat.com>; Lennart Poettering <mzxreary@0pointer.de>; Peter Matthias
> <matthias.peter@bsi.bund.de>; Marcelo Henrique Cerri <marcelo.cerri@canonical.com>; Neil Horman <nhorman@redhat.com>;
> Randy Dunlap <rdunlap@infradead.org>; Julia Lawall <julia.lawall@inria.fr>; Dan Carpenter <dan.carpenter@oracle.com>; Andy Lavr
> <andy.lavr@gmail.com>; Eric Biggers <ebiggers@kernel.org>; Jason A. Donenfeld <Jason@zx2c4.com>; Stephan Müller
> <smueller@chronox.de>; Petr Tesarik <ptesarik@suse.cz>
> Subject: Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance
>
> <<< External Email >>>
> Almost two weeks passed and these are the "relevant" replies:
>
> Jason personally does not like FIPS, and is afraid of
> "subpar crypto". Albeit this patch set strictly isn't about
> crypto at all; the crypto subsystem is in the unlucky position
> to just depend on a good entropy source.
>
IMHO, Jason's statement is completely silly and solely based on some personal beef.
Obviously, the _ability_ to be compliant with FIPS testing does not preclude the use
of non-FIPS crypto, in case you should choose not to trust any of the FIPS recommended
implementations.

Fact of the matter is, many application areas (including but not limited to defence,
industrial automation, automotive, aero space, ...) have a hard a hard requirement on
FIPS certification. So not supporting that would either rule out using Linux altogether,
or steer them towards out-of-tree solutions.

And just running tests on your entropy source can't possibly be a bad thing anyway,
especially if you can configure it out if don't need or want to have it.

> Greg claims that Linux (kernel) isn't about choice, which is clearly
> wrong.
>
Well, I'm not going to argue with Greg about that ;-)

> And this is all ???
>
> There are options for stack protection. I can see bounds checking
> and other sanity checks all over the place. And doing a similar thing
> on entropy sources is a problem?
>
> Admittedly, if entropy sources fail, the kernel will happily remain
> running. No bad immediate effects in userland will arise. Only some
> cryptographic algorithms, otherwise very decent, will run on
> unneccessarily weak keys, probably causing some real-world problems.
> Does anybody care?
> The NIST and the BSI do, but that does not mean their solutions are
> automatically wrong or backdoored.
>
> There is now a well layed-out scheme to ensure quality randomness,
> and a lot of work here has been put into its implementation.
>
> Would some maintainer please comment on potential problems or
> shortcomings? Otherwise a "Thanks, applied" would be appropriate, IMO.
>
> Torsten

Regards,
Pascal van Leeuwen
Silicon IP Architect Multi-Protocol Engines, Rambus Security
Rambus ROTW Holding BV
+31-73 6581953

Note: The Inside Secure/Verimatrix Silicon IP team was recently acquired by Rambus.
Please be so kind to update your e-mail address book with my new e-mail address.

** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying, forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **

Rambus Inc.<http://www.rambus.com>

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Stephan Mueller]

Am Freitag, 2. Oktober 2020, 15:15:55 CEST schrieb Willy Tarreau:

Hi Willy,

> > And this is all ???
> 
> Possibly a lot of people got used to seeing the numerous versions
> and are less attentive to new series, it's possible that your message
> will wake everyone up.

I think that points to my patch series. My patch series which provide a 
complete separate, API and ABI compliant drop in replacement of /dev/random, 
nobody from the gatekeepers cared to even answer. It would not touch the 
existing code.

After waiting some time without changing the code (e.g. after Andi Lutomirski 
commented), I got no answer at all from the gatekeepers, not even any 
indication in what direction I should move if something was not desired in the 
patch series.

Thus I continued adding the features that I think are necessary and for which 
I received comments from mathematicians. What else should I do?

With the patch set v35 of my patch series, I see all my goals finally 
achieved at I expect the code to be stable from here on. The last one was the 
hardest: to get rid of all non-cryptographic conditioning operations and yet 
retain performance en par or even superior to the existing /dev/random 
implementation.

Ciao
Stephan

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Greg Kroah-Hartman]

On Fri, Oct 02, 2020 at 01:35:18PM +0000, Van Leeuwen, Pascal wrote:
> ** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying, forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **

As per my legal department requests, this is now ignored and deleted on
my system...

Hint, it's not a valid footer for public mailing lists...

greg k-h

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Torsten Duwe]

On Fri, Oct 02, 2020 at 03:33:58PM +0200, Greg Kroah-Hartman wrote:
> On Fri, Oct 02, 2020 at 03:15:55PM +0200, Willy Tarreau wrote:
> > On Fri, Oct 02, 2020 at 02:38:36PM +0200, Torsten Duwe wrote:
> > > Almost two weeks passed and these are the "relevant" replies:
> > > 
> > > Jason personally does not like FIPS, and is afraid of
> > > "subpar crypto". Albeit this patch set strictly isn't about
> > > crypto at all; the crypto subsystem is in the unlucky position
> > > to just depend on a good entropy source.
> > > 
> > > Greg claims that Linux (kernel) isn't about choice, which is clearly
> > > wrong.
> > 
> > I think there's a small misunderstanding here, my understanding is
> > that for quite a while, the possibilities offered by the various
> > random subsystems or their proposed derivative used to range from
> > "you have to choose between a fast system that may be vulnerable
> > to some attacks, a system that might not be vulnerable to certain
> > attacks but might not always boot, or a slow system not vulnerable
> > to certain attacks". Greg's point seems to be that if we add an
> > option, it means it's yet another tradeoff between these possibilities
> > and that someone will still not be happy at the end of the chain. If
> > the proposed solution covers everything at once (performance,
> > reliability, unpredictability), then there probably is no more reason
> > for keeping alternate solutions at all, hence there's no need to give
> > the user the choice between multiple options when only one is known
> > to always be valid. At least that's how I see it and it makes sense
> > to me.
> 
> Thanks for spelling it out in much more detail than I was willing to :)

I assume you're not trying to pull the discussion off-topic. The one and
only choice here is that some people believe in NIST and certifications.
Yes, others don't, no problem either. The former folks boot with fips=1,
that's it. Those people are usually certain about their decision.

That option is about to break, for reasons I stated previously. This patch
set is to introduce the now-missing pieces. One thing worth to discuss here
would be whether people not so security conscious should benefit from the
sanity checks as well. IMHO they should, because, as Willy explained, stick
with the option that's always valid.

My disappointment was that _none_ of the maintaners had an on-topic,
technical remark. I get the impression some read "FIPS" and stop, regardless
of the actual functionality.

	Torsten

RE: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Van Leeuwen, Pascal]

> -----Original Message-----
> From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Sent: Friday, October 2, 2020 4:04 PM
> To: Van Leeuwen, Pascal <pvanleeuwen@rambus.com>
> Cc: Torsten Duwe <duwe@lst.de>; Theodore Y. Ts'o <tytso@mit.edu>; linux-crypto@vger.kernel.org; Nicolai Stange
> <nstange@suse.de>; LKML <linux-kernel@vger.kernel.org>; Arnd Bergmann <arnd@arndb.de>; Eric W. Biederman
> <ebiederm@xmission.com>; Alexander E. Patrakov <patrakov@gmail.com>; Ahmed S. Darwish <darwish.07@gmail.com>; Willy
> Tarreau <w@1wt.eu>; Matthew Garrett <mjg59@srcf.ucam.org>; Vito Caputo <vcaputo@pengaru.com>; Andreas Dilger
> <adilger.kernel@dilger.ca>; Jan Kara <jack@suse.cz>; Ray Strode <rstrode@redhat.com>; William Jon McCann <mccann@jhu.edu>;
> zhangjs <zachary@baishancloud.com>; Andy Lutomirski <luto@kernel.org>; Florian Weimer <fweimer@redhat.com>; Lennart
> Poettering <mzxreary@0pointer.de>; Peter Matthias <matthias.peter@bsi.bund.de>; Marcelo Henrique Cerri
> <marcelo.cerri@canonical.com>; Neil Horman <nhorman@redhat.com>; Randy Dunlap <rdunlap@infradead.org>; Julia Lawall
> <julia.lawall@inria.fr>; Dan Carpenter <dan.carpenter@oracle.com>; Andy Lavr <andy.lavr@gmail.com>; Eric Biggers
> <ebiggers@kernel.org>; Jason A. Donenfeld <Jason@zx2c4.com>; Stephan Müller <smueller@chronox.de>; Petr Tesarik
> <ptesarik@suse.cz>
> Subject: Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance
>
> <<< External Email >>>
> On Fri, Oct 02, 2020 at 01:35:18PM +0000, Van Leeuwen, Pascal wrote:
> > ** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is
> confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying,
> forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **
>
> As per my legal department requests, this is now ignored and deleted on
> my system...
>
> Hint, it's not a valid footer for public mailing lists...
>
> greg k-h
It's automatically added by our company mail server ... not something I can control at all :-(
And using some external SMTP server would not pass our firewall.
So free webmail would be my only alternative, but I have a thorough dislike for web-based
tools, as I have yet to come across one with an even remotely acceptable user interface.

Regards,
Pascal van Leeuwen
Silicon IP Architect Multi-Protocol Engines, Rambus Security
Rambus ROTW Holding BV
+31-73 6581953

Note: The Inside Secure/Verimatrix Silicon IP team was recently acquired by Rambus.
Please be so kind to update your e-mail address book with my new e-mail address.

** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying, forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **

Rambus Inc.<http://www.rambus.com>

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Greg Kroah-Hartman]

On Fri, Oct 02, 2020 at 02:34:44PM +0000, Van Leeuwen, Pascal wrote:
> 
> 
> 
> > -----Original Message-----
> > From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Sent: Friday, October 2, 2020 4:04 PM
> > To: Van Leeuwen, Pascal <pvanleeuwen@rambus.com>
> > Cc: Torsten Duwe <duwe@lst.de>; Theodore Y. Ts'o <tytso@mit.edu>; linux-crypto@vger.kernel.org; Nicolai Stange
> > <nstange@suse.de>; LKML <linux-kernel@vger.kernel.org>; Arnd Bergmann <arnd@arndb.de>; Eric W. Biederman
> > <ebiederm@xmission.com>; Alexander E. Patrakov <patrakov@gmail.com>; Ahmed S. Darwish <darwish.07@gmail.com>; Willy
> > Tarreau <w@1wt.eu>; Matthew Garrett <mjg59@srcf.ucam.org>; Vito Caputo <vcaputo@pengaru.com>; Andreas Dilger
> > <adilger.kernel@dilger.ca>; Jan Kara <jack@suse.cz>; Ray Strode <rstrode@redhat.com>; William Jon McCann <mccann@jhu.edu>;
> > zhangjs <zachary@baishancloud.com>; Andy Lutomirski <luto@kernel.org>; Florian Weimer <fweimer@redhat.com>; Lennart
> > Poettering <mzxreary@0pointer.de>; Peter Matthias <matthias.peter@bsi.bund.de>; Marcelo Henrique Cerri
> > <marcelo.cerri@canonical.com>; Neil Horman <nhorman@redhat.com>; Randy Dunlap <rdunlap@infradead.org>; Julia Lawall
> > <julia.lawall@inria.fr>; Dan Carpenter <dan.carpenter@oracle.com>; Andy Lavr <andy.lavr@gmail.com>; Eric Biggers
> > <ebiggers@kernel.org>; Jason A. Donenfeld <Jason@zx2c4.com>; Stephan Müller <smueller@chronox.de>; Petr Tesarik
> > <ptesarik@suse.cz>
> > Subject: Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance
> >
> > <<< External Email >>>
> > On Fri, Oct 02, 2020 at 01:35:18PM +0000, Van Leeuwen, Pascal wrote:
> > > ** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is
> > confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying,
> > forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **
> >
> > As per my legal department requests, this is now ignored and deleted on
> > my system...
> >
> > Hint, it's not a valid footer for public mailing lists...
> >
> > greg k-h
> It's automatically added by our company mail server ... not something I can control at all :-(

Then your company can not contribute in Linux kernel development, as
this is obviously not allowed by such a footer.

Please work with your IT and legal department to fix this.

thanks,

greg k-h

RE: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Van Leeuwen, Pascal]

> -----Original Message-----
> From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Sent: Friday, October 2, 2020 5:13 PM
> To: Van Leeuwen, Pascal <pvanleeuwen@rambus.com>
> Cc: Torsten Duwe <duwe@lst.de>; Theodore Y. Ts'o <tytso@mit.edu>; linux-crypto@vger.kernel.org; Nicolai Stange
> <nstange@suse.de>; LKML <linux-kernel@vger.kernel.org>; Arnd Bergmann <arnd@arndb.de>; Eric W. Biederman
> <ebiederm@xmission.com>; Alexander E. Patrakov <patrakov@gmail.com>; Ahmed S. Darwish <darwish.07@gmail.com>; Willy
> Tarreau <w@1wt.eu>; Matthew Garrett <mjg59@srcf.ucam.org>; Vito Caputo <vcaputo@pengaru.com>; Andreas Dilger
> <adilger.kernel@dilger.ca>; Jan Kara <jack@suse.cz>; Ray Strode <rstrode@redhat.com>; William Jon McCann <mccann@jhu.edu>;
> zhangjs <zachary@baishancloud.com>; Andy Lutomirski <luto@kernel.org>; Florian Weimer <fweimer@redhat.com>; Lennart
> Poettering <mzxreary@0pointer.de>; Peter Matthias <matthias.peter@bsi.bund.de>; Marcelo Henrique Cerri
> <marcelo.cerri@canonical.com>; Neil Horman <nhorman@redhat.com>; Randy Dunlap <rdunlap@infradead.org>; Julia Lawall
> <julia.lawall@inria.fr>; Dan Carpenter <dan.carpenter@oracle.com>; Andy Lavr <andy.lavr@gmail.com>; Eric Biggers
> <ebiggers@kernel.org>; Jason A. Donenfeld <Jason@zx2c4.com>; Stephan Müller <smueller@chronox.de>; Petr Tesarik
> <ptesarik@suse.cz>
> Subject: Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance
>
> <<< External Email >>>
> On Fri, Oct 02, 2020 at 02:34:44PM +0000, Van Leeuwen, Pascal wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > > Sent: Friday, October 2, 2020 4:04 PM
> > > To: Van Leeuwen, Pascal <pvanleeuwen@rambus.com>
> > > Cc: Torsten Duwe <duwe@lst.de>; Theodore Y. Ts'o <tytso@mit.edu>; linux-crypto@vger.kernel.org; Nicolai Stange
> > > <nstange@suse.de>; LKML <linux-kernel@vger.kernel.org>; Arnd Bergmann <arnd@arndb.de>; Eric W. Biederman
> > > <ebiederm@xmission.com>; Alexander E. Patrakov <patrakov@gmail.com>; Ahmed S. Darwish <darwish.07@gmail.com>; Willy
> > > Tarreau <w@1wt.eu>; Matthew Garrett <mjg59@srcf.ucam.org>; Vito Caputo <vcaputo@pengaru.com>; Andreas Dilger
> > > <adilger.kernel@dilger.ca>; Jan Kara <jack@suse.cz>; Ray Strode <rstrode@redhat.com>; William Jon McCann
> <mccann@jhu.edu>;
> > > zhangjs <zachary@baishancloud.com>; Andy Lutomirski <luto@kernel.org>; Florian Weimer <fweimer@redhat.com>; Lennart
> > > Poettering <mzxreary@0pointer.de>; Peter Matthias <matthias.peter@bsi.bund.de>; Marcelo Henrique Cerri
> > > <marcelo.cerri@canonical.com>; Neil Horman <nhorman@redhat.com>; Randy Dunlap <rdunlap@infradead.org>; Julia Lawall
> > > <julia.lawall@inria.fr>; Dan Carpenter <dan.carpenter@oracle.com>; Andy Lavr <andy.lavr@gmail.com>; Eric Biggers
> > > <ebiggers@kernel.org>; Jason A. Donenfeld <Jason@zx2c4.com>; Stephan Müller <smueller@chronox.de>; Petr Tesarik
> > > <ptesarik@suse.cz>
> > > Subject: Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance
> > >
> > > <<< External Email >>>
> > > On Fri, Oct 02, 2020 at 01:35:18PM +0000, Van Leeuwen, Pascal wrote:
> > > > ** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is
> > > confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying,
> > > forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **
> > >
> > > As per my legal department requests, this is now ignored and deleted on
> > > my system...
> > >
> > > Hint, it's not a valid footer for public mailing lists...
> > >
> > > greg k-h
> > It's automatically added by our company mail server ... not something I can control at all :-(
>
> Then your company can not contribute in Linux kernel development, as
> this is obviously not allowed by such a footer.
>
Interesting, this has never been raised as a problem until today ...
Going back through my mail archive, it looks like they started automatically adding that some
3 months ago. Not that they informed anyone about that, it just silently happened.

> Please work with your IT and legal department to fix this.
>
Eh ... Greg ... that's not how that works in the real world. In the real world, legal and IT lay
down the law and you just comply with that (or hack your way around it, if you can ;-).

I'm already fighting the good fight trying to keep control of my development machines
because IT would just love to get rid of those (since not under IT control .... oh dear ...)
And obviously, you cannot do kernel development on a machine without root access.
It's annoying enough already to require IT support to provide explicit permission to open
the task manager on my own company laptop ... grmbl.

>
> thanks,
>
> greg k-h

Regards,
Pascal van Leeuwen
Silicon IP Architect Multi-Protocol Engines, Rambus Security
Rambus ROTW Holding BV
+31-73 6581953

Note: The Inside Secure/Verimatrix Silicon IP team was recently acquired by Rambus.
Please be so kind to update your e-mail address book with my new e-mail address.


** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying, forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **

Rambus Inc.<http://www.rambus.com>

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Randy Dunlap]

On 10/2/20 8:39 AM, Van Leeuwen, Pascal wrote:
>> -----Original Message-----
>> From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>> Sent: Friday, October 2, 2020 5:13 PM
>> To: Van Leeuwen, Pascal <pvanleeuwen@rambus.com>
>> Cc: Torsten Duwe <duwe@lst.de>; Theodore Y. Ts'o <tytso@mit.edu>; linux-crypto@vger.kernel.org; Nicolai Stange
>> <nstange@suse.de>; LKML <linux-kernel@vger.kernel.org>; Arnd Bergmann <arnd@arndb.de>; Eric W. Biederman
>> <ebiederm@xmission.com>; Alexander E. Patrakov <patrakov@gmail.com>; Ahmed S. Darwish <darwish.07@gmail.com>; Willy
>> Tarreau <w@1wt.eu>; Matthew Garrett <mjg59@srcf.ucam.org>; Vito Caputo <vcaputo@pengaru.com>; Andreas Dilger
>> <adilger.kernel@dilger.ca>; Jan Kara <jack@suse.cz>; Ray Strode <rstrode@redhat.com>; William Jon McCann <mccann@jhu.edu>;
>> zhangjs <zachary@baishancloud.com>; Andy Lutomirski <luto@kernel.org>; Florian Weimer <fweimer@redhat.com>; Lennart
>> Poettering <mzxreary@0pointer.de>; Peter Matthias <matthias.peter@bsi.bund.de>; Marcelo Henrique Cerri
>> <marcelo.cerri@canonical.com>; Neil Horman <nhorman@redhat.com>; Randy Dunlap <rdunlap@infradead.org>; Julia Lawall
>> <julia.lawall@inria.fr>; Dan Carpenter <dan.carpenter@oracle.com>; Andy Lavr <andy.lavr@gmail.com>; Eric Biggers
>> <ebiggers@kernel.org>; Jason A. Donenfeld <Jason@zx2c4.com>; Stephan Müller <smueller@chronox.de>; Petr Tesarik
>> <ptesarik@suse.cz>
>> Subject: Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance
>>
>> <<< External Email >>>
>> On Fri, Oct 02, 2020 at 02:34:44PM +0000, Van Leeuwen, Pascal wrote:
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>>>> Sent: Friday, October 2, 2020 4:04 PM
>>>> To: Van Leeuwen, Pascal <pvanleeuwen@rambus.com>
>>>> Cc: Torsten Duwe <duwe@lst.de>; Theodore Y. Ts'o <tytso@mit.edu>; linux-crypto@vger.kernel.org; Nicolai Stange
>>>> <nstange@suse.de>; LKML <linux-kernel@vger.kernel.org>; Arnd Bergmann <arnd@arndb.de>; Eric W. Biederman
>>>> <ebiederm@xmission.com>; Alexander E. Patrakov <patrakov@gmail.com>; Ahmed S. Darwish <darwish.07@gmail.com>; Willy
>>>> Tarreau <w@1wt.eu>; Matthew Garrett <mjg59@srcf.ucam.org>; Vito Caputo <vcaputo@pengaru.com>; Andreas Dilger
>>>> <adilger.kernel@dilger.ca>; Jan Kara <jack@suse.cz>; Ray Strode <rstrode@redhat.com>; William Jon McCann
>> <mccann@jhu.edu>;
>>>> zhangjs <zachary@baishancloud.com>; Andy Lutomirski <luto@kernel.org>; Florian Weimer <fweimer@redhat.com>; Lennart
>>>> Poettering <mzxreary@0pointer.de>; Peter Matthias <matthias.peter@bsi.bund.de>; Marcelo Henrique Cerri
>>>> <marcelo.cerri@canonical.com>; Neil Horman <nhorman@redhat.com>; Randy Dunlap <rdunlap@infradead.org>; Julia Lawall
>>>> <julia.lawall@inria.fr>; Dan Carpenter <dan.carpenter@oracle.com>; Andy Lavr <andy.lavr@gmail.com>; Eric Biggers
>>>> <ebiggers@kernel.org>; Jason A. Donenfeld <Jason@zx2c4.com>; Stephan Müller <smueller@chronox.de>; Petr Tesarik
>>>> <ptesarik@suse.cz>
>>>> Subject: Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance
>>>>
>>>> <<< External Email >>>
>>>> On Fri, Oct 02, 2020 at 01:35:18PM +0000, Van Leeuwen, Pascal wrote:
>>>>> ** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is
>>>> confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying,
>>>> forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **
>>>>
>>>> As per my legal department requests, this is now ignored and deleted on
>>>> my system...
>>>>
>>>> Hint, it's not a valid footer for public mailing lists...
>>>>
>>>> greg k-h
>>> It's automatically added by our company mail server ... not something I can control at all :-(
>>
>> Then your company can not contribute in Linux kernel development, as
>> this is obviously not allowed by such a footer.
>>
> Interesting, this has never been raised as a problem until today ...
> Going back through my mail archive, it looks like they started automatically adding that some
> 3 months ago. Not that they informed anyone about that, it just silently happened.
> 
>> Please work with your IT and legal department to fix this.
>>
> Eh ... Greg ... that's not how that works in the real world. In the real world, legal and IT lay
> down the law and you just comply with that (or hack your way around it, if you can ;-).

That's how it worked at $big_companies that I have worked at.

If it's a company/business requirement that you do Linux kernel development work, (is it?)
then they should make that possible on internal systems or give you access to
external email server(s).

> I'm already fighting the good fight trying to keep control of my development machines
> because IT would just love to get rid of those (since not under IT control .... oh dear ...)
> And obviously, you cannot do kernel development on a machine without root access.
> It's annoying enough already to require IT support to provide explicit permission to open
> the task manager on my own company laptop ... grmbl.
> 
>>
>> thanks,
>>
>> greg k-h
> 
> Regards,
> Pascal van Leeuwen
> Silicon IP Architect Multi-Protocol Engines, Rambus Security
> Rambus ROTW Holding BV
> +31-73 6581953
> 
> Note: The Inside Secure/Verimatrix Silicon IP team was recently acquired by Rambus.
> Please be so kind to update your e-mail address book with my new e-mail address.
> 
> 
> ** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying, forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **
> 
> Rambus Inc.<http://www.rambus.com>
> 


-- 
~Randy

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Theodore Y. Ts'o]

On Fri, Oct 02, 2020 at 03:39:35PM +0000, Van Leeuwen, Pascal wrote:
> > Then your company can not contribute in Linux kernel development, as
> > this is obviously not allowed by such a footer.
> >
> Interesting, this has never been raised as a problem until today ...
> Going back through my mail archive, it looks like they started automatically adding that some
> 3 months ago. Not that they informed anyone about that, it just silently happened.

So use a private e-mail address (e.g., at fastmail.fm if you don't
want to run your mail server) and then tunnel out SMTP requests using
ssh.  It's not hard.  :-)

I've worked a multiple $BIG_COMPANY's, and I've been doing this for
decades.  It's also helpful when I need to send e-mails from
conference networks from my laptop....

						- Ted

RE: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Van Leeuwen, Pascal]

> -----Original Message-----
> From: Theodore Y. Ts'o <tytso@mit.edu>
> Sent: Friday, October 2, 2020 8:14 PM
> To: Van Leeuwen, Pascal <pvanleeuwen@rambus.com>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>; Torsten Duwe <duwe@lst.de>; linux-crypto@vger.kernel.org; Nicolai Stange
> <nstange@suse.de>; LKML <linux-kernel@vger.kernel.org>; Arnd Bergmann <arnd@arndb.de>; Eric W. Biederman
> <ebiederm@xmission.com>; Alexander E. Patrakov <patrakov@gmail.com>; Ahmed S. Darwish <darwish.07@gmail.com>; Willy
> Tarreau <w@1wt.eu>; Matthew Garrett <mjg59@srcf.ucam.org>; Vito Caputo <vcaputo@pengaru.com>; Andreas Dilger
> <adilger.kernel@dilger.ca>; Jan Kara <jack@suse.cz>; Ray Strode <rstrode@redhat.com>; William Jon McCann <mccann@jhu.edu>;
> zhangjs <zachary@baishancloud.com>; Andy Lutomirski <luto@kernel.org>; Florian Weimer <fweimer@redhat.com>; Lennart
> Poettering <mzxreary@0pointer.de>; Peter Matthias <matthias.peter@bsi.bund.de>; Marcelo Henrique Cerri
> <marcelo.cerri@canonical.com>; Neil Horman <nhorman@redhat.com>; Randy Dunlap <rdunlap@infradead.org>; Julia Lawall
> <julia.lawall@inria.fr>; Dan Carpenter <dan.carpenter@oracle.com>; Andy Lavr <andy.lavr@gmail.com>; Eric Biggers
> <ebiggers@kernel.org>; Jason A. Donenfeld <Jason@zx2c4.com>; Stephan Müller <smueller@chronox.de>; Petr Tesarik
> <ptesarik@suse.cz>
> Subject: Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance
>
> <<< External Email >>>
> On Fri, Oct 02, 2020 at 03:39:35PM +0000, Van Leeuwen, Pascal wrote:
> > > Then your company can not contribute in Linux kernel development, as
> > > this is obviously not allowed by such a footer.
> > >
> > Interesting, this has never been raised as a problem until today ...
> > Going back through my mail archive, it looks like they started automatically adding that some
> > 3 months ago. Not that they informed anyone about that, it just silently happened.
>
> So use a private e-mail address (e.g., at fastmail.fm if you don't
> want to run your mail server) and then tunnel out SMTP requests using
> ssh.  It's not hard.  :-)
>
Actually, for the last patches I sent I already had to tunnel them over some 3rd
party SMTP-over-HTTPS service because of our firewall blocking access to
the Gmail SMTP server I previously used for that :-(

I guess tunnelling over SSH is another option, although I have no idea how to
do such a thing (didn't know it was possible). At the end of the day, I am not
a software guy, so I'm not _that_ much into these kinds of things ...

That doesn't work for _regular_ mail, though, as I would not be able to setup
a mail client for that. I can't install anything and I can't even touch the settings
of my Outlook client :-(

For incoming mail I'm actually bulk forwarding the mailing list through my
Gmail account now because our mail server stopped accepting it directly.
WIth POP3 and IMAP being blocked, I still need to find some way to receive
patches without our Exchange server fubarring them though (tips are welcome!).
Right now the only solution I have is fetch them from my home PC and take
them to work on a USB stick. Welcome to 2020 ...

> I've worked a multiple $BIG_COMPANY's, and I've been doing this for
> decades.  It's also helpful when I need to send e-mails from
> conference networks from my laptop....
>
>

Regards,
Pascal van Leeuwen
Silicon IP Architect Multi-Protocol Engines, Rambus Security
Rambus ROTW Holding BV
+31-73 6581953

Note: The Inside Secure/Verimatrix Silicon IP team was recently acquired by Rambus.
Please be so kind to update your e-mail address book with my new e-mail address.
- Ted


** This message and any attachments are for the sole use of the intended recipient(s). It may contain information that is confidential and privileged. If you are not the intended recipient of this message, you are prohibited from printing, copying, forwarding or saving it. Please delete the message and attachments and notify the sender immediately. **

Rambus Inc.<http://www.rambus.com>

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Eric Biggers]

On Fri, Oct 02, 2020 at 02:38:36PM +0200, Torsten Duwe wrote:
> Almost two weeks passed and these are the "relevant" replies:
> 
> Jason personally does not like FIPS, and is afraid of
> "subpar crypto". Albeit this patch set strictly isn't about
> crypto at all; the crypto subsystem is in the unlucky position
> to just depend on a good entropy source.
> 
> Greg claims that Linux (kernel) isn't about choice, which is clearly
> wrong.
> 
> And this is all ???
> 
> There are options for stack protection. I can see bounds checking
> and other sanity checks all over the place. And doing a similar thing
> on entropy sources is a problem?
> 
> Admittedly, if entropy sources fail, the kernel will happily remain
> running. No bad immediate effects in userland will arise. Only some
> cryptographic algorithms, otherwise very decent, will run on
> unneccessarily weak keys, probably causing some real-world problems.
> Does anybody care?
> The NIST and the BSI do, but that does not mean their solutions are
> automatically wrong or backdoored.
> 
> There is now a well layed-out scheme to ensure quality randomness,
> and a lot of work here has been put into its implementation.
> 
> Would some maintainer please comment on potential problems or
> shortcomings? Otherwise a "Thanks, applied" would be appropriate, IMO.
> 

Well, very people are experts in the Linux RNG *and* have time to review large
patchsets, especially when three people are all proposing conflicting changes.
And those that might be able to review these patches aren't necessarily
interested in compliance with particular government standards.

Note that having multiple RNG implementations would cause fragmentation, more
maintenance burden, etc.  So IMO, that should be a last resort.  Instead we
should try to find an implementation that works for everyone.  I.e., at least to
me, Nicolai's patchset seems more on the right track than Stephan's patchset...

However, not everyone cares about "compliance".  So any changes for "compliance"
either need to have a real technical argument for making the change, *or* need
to be optional (e.g. controlled by fips_enabled).

AFAICS, this patchset mostly just talks about NIST SP800-90B compliance, and
doesn't make clear whether the changes make the RNG better, worse, or the same
from an actual technical perspective.

If that was properly explained, and if the answer was "better" or at least
"not worse", I expect that people would be more interested.

- Eric

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Stephan Mueller]

Am Mittwoch, 7. Oktober 2020, 06:24:09 CEST schrieb Eric Biggers:

Hi Eric,
> 
> Note that having multiple RNG implementations would cause fragmentation,
> more maintenance burden, etc.  So IMO, that should be a last resort. 
> Instead we should try to find an implementation that works for everyone. 
> I.e., at least to me, Nicolai's patchset seems more on the right track than
> Stephan's patchset...

Thank you for sharing your considerations.

If you say that only one implementation should be there, I am wondering why 
not considering an implementation that as significant advantages over the 
existing implementation as outlined in my cover letter to patch v35. In the 
default configuration, it compiles no code at all that has any bearing on 
government standards. Yet it has a more cryptographic sound approach to handle 
entropy. In addition is meant to be extensible allowing each user to pick and 
chose what he wants. Yet, users who do not want these extensions should not 
suffer from it (neither performance-wise, nor should they suffer from an 
unnecessary complex code that builds all options into one C file).

And speaking of fragmentation, if it is not *possible* to allow users to pick 
what they want and need (and yes, in some parts of the world or for some users 
these government standards are simply a necessity), we surely invite 
fragmentation. In the LRNG, I tried to have all operations critical to entropy 
compression and random number generation modularized so that the a can be 
replaced or extended if needed without fragmentation.

PS: The reason why I started the LRNG was not government standards, but the 
result of performing two studies. The one study was about entropy in 
virtualized environment which showed that we have significant entropy in 
virtual environments and yet the existing /dev/random implementation thinks 
there is much less available. Another study I maintain for years also shows 
that the entire entropy collection and heuristic on bare metal systems is also 
in need of advancements. Initially I provided patches to the existing /dev/
random implementation, but basically all were silently ignored.

Ciao
Stephan

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Nicolai Stange]

Eric Biggers <ebiggers@kernel.org> writes:

> On Fri, Oct 02, 2020 at 02:38:36PM +0200, Torsten Duwe wrote:
>>
>> Would some maintainer please comment on potential problems or
>> shortcomings?
>> 
>
> Well, very people are experts in the Linux RNG *and* have time to review large
> patchsets, especially when three people are all proposing conflicting changes.
> And those that might be able to review these patches aren't necessarily
> interested in compliance with particular government standards.

To make it clear: I'm personally not really enthusiastic about some of
the restrictions imposed by SP800-90 either and Jason certainly has a
point with his concerns about "subpar crypto" ([1]). However, at the
same time I'm acknowledging that for some users FIPS compliance is
simply a necessity and I don't see a strong reason why that shouldn't be
supported, if doable without negatively affecting !fips_enabled users.


> Note that having multiple RNG implementations would cause fragmentation, more
> maintenance burden, etc.  So IMO, that should be a last resort.  Instead we
> should try to find an implementation that works for everyone.  I.e., at least to
> me, Nicolai's patchset seems more on the right track than Stephan's patchset...

I suppose that this concern about fragmentation is among the main
reasons for reservations against Stephan's LRNG patchset and that's why
I posted this RFC series here for comparison purposes. But note that, as
said ([2]), it's incomplete and the only intent was to provide at least
a rough idea on what it would take to move the current /dev/random
implementation towards SP800-90 -- I was hoping for either a hard NACK
or something along the lines of "maybe, go ahead and let's see".


> However, not everyone cares about "compliance".  So any changes for "compliance"
> either need to have a real technical argument for making the change, *or* need
> to be optional (e.g. controlled by fips_enabled).

Fully agreed.


> AFAICS, this patchset mostly just talks about NIST SP800-90B compliance, and
> doesn't make clear whether the changes make the RNG better, worse, or the same
> from an actual technical perspective.
>
> If that was properly explained, and if the answer was "better" or at least
> "not worse", I expect that people would be more interested.

The goal was not to negatively affect !fips_enabled users, but as
outlined in the cover letter ([2]), a performance impact had been
measured on ARMv7. This probably isn't something which couldn't get
sorted out, but I see no point in doing it at this stage, because
- there's still quite some stuff missing for full SP800-90 compliance
  anyway, c.f. the overview at the end of [2] and
- such optimizations would have bloated this patchset even more,
  e.g. for making fips_enabled a static_key, which should certainly go
  into a separate series.

User visible effects set aside, an obvious downside of SP800-90
compliance would be the increase in code size and the associated
maintenance burden.

That being said, I can imagine that those boot health tests could also
get enabled for !fips_enabled users in the future, if wanted: rather
than inhibiting /dev/random output on failure, a warning would get
logged instead.  Whether or not this would be seen as an improvement
is for others to judge though.

Thanks,

Nicolai


[1] https://lkml.kernel.org/r/CAHmME9rMXORFXtwDAc8yxj+h9gytJj6DpvCxA-JMAAgyOP+5Yw@mail.gmail.com
[2] https://lkml.kernel.org/r/20200921075857.4424-1-nstange@suse.de

-- 
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg), GF: Felix Imendörffer

Re: [DISCUSSION PATCH 00/41] random: possible ways towards NIST SP800-90B compliance

[Torsten Duwe]

On Fri, Oct 02, 2020 at 03:56:28PM +0200, Stephan Mueller wrote:
> Am Freitag, 2. Oktober 2020, 15:15:55 CEST schrieb Willy Tarreau:
> 
> Hi Willy,
> 
> > > And this is all ???
> > 
> > Possibly a lot of people got used to seeing the numerous versions
> > and are less attentive to new series, it's possible that your message
> > will wake everyone up.
> 
> I think that points to my patch series. My patch series which provide a 
> complete separate, API and ABI compliant drop in replacement of /dev/random, 
> nobody from the gatekeepers cared to even answer. It would not touch the 
> existing code.
> 
> After waiting some time without changing the code (e.g. after Andi Lutomirski 
> commented), I got no answer at all from the gatekeepers, not even any 
> indication in what direction I should move if something was not desired in the 
> patch series.
> 
> Thus I continued adding the features that I think are necessary and for which 
> I received comments from mathematicians. What else should I do?
> 
> With the patch set v35 of my patch series, I see all my goals finally 
> achieved at I expect the code to be stable from here on. The last one was the 
> hardest: to get rid of all non-cryptographic conditioning operations and yet 
> retain performance en par or even superior to the existing /dev/random 
> implementation.

Would you mind to resend it here, for a comparison?

	Torsten

[PATCH v36 00/13] /dev/random - a new approach

[Stephan Müller]

Hi,

The following patch set provides a different approach to /dev/random which
is called Linux Random Number Generator (LRNG) to collect entropy within
the Linux kernel. It provides the same API and ABI and can be used as a
drop-in replacement.

The LRNG implements at least all features of the existing /dev/random such as
NUMA-node-local DRNGs. Patches 1 through 3 provide the code that is feature-
identical. The following advantages compared to the existing /dev/random
implementation are present:

* Sole use of crypto for data processing:

 - Exclusive use of a hash operation for conditioning entropy data with
   a clear mathematical description as given in [2] section 2.2 -
   non-cryptographic operations like LFSR are not used.

 - The LRNG uses only properly defined and implemented cryptographic
   algorithms unlike the use of the SHA-1 transformation in the existing
   /dev/random implementation.

 - Hash operations use NUMA-node-local hash instances to benefit large
   parallel systems.

 - LRNG uses limited number of data post-processing steps as documented in
   [2] section 2.2 compared to the large variation of different
   post-processing steps in the existing /dev/random implementation that
   have no apparent mathematical description (see [2] section 4.5).

* Performance

 - Faster by up to 75% in the critical code path of the interrupt handler
   depending on data collection size configurable at kernel compile time -
   the default is about equal in performance with existing /dev/random as
   outlined in [2] section 4.2.

 - Configurable data collection sizes to accommodate small environments
   and big environments via CONFIG_LRNG_COLLECTION_SIZE.

 - Entropy collection using an almost never contended lock to benefit
   large parallel systems – worst case rate of contention is the number
   of DRNG reseeds, usually the number of potential contentions per 10
   minutes is equal to number of NUMA nodes.

 - ChaCha20 DRNG is significantly faster as implemented in the existing
   /dev/random as demonstrated with [2] table 2.

 - Faster entropy collection during boot time to reach fully seeded
   level, including on virtual systems or systems with SSDs as outlined
   in [2] section 4.1.

* Testing

 - Availablility of run-time health tests of the raw unconditioned
   noise source to identify degradation of the available entropy as
   documented in [2] section 2.5.4. Such health tests are important
   today due to virtual machine monitors reducing the resolution of
   or disabling the high-resolution timer.

 - Heuristic entropy estimation is based on quantitative measurements
   and analysis following SP800-90B and not on coincidental
   underestimation of entropy applied by the existing /dev/random as
   outlined in [4] section 4.4.

 - Power-on self tests for critical deterministic components (ChaCha20
   DRNG, software hash implementation, and entropy collection logic)
   not already covered by power-up tests of the kernel crypto API as
   documented in [2] section 2.14.

 - Availability of test interfaces for all operational stages of the
   LRNG including boot-time raw entropy event data sampling as outlined
   in [2] section 2.15.

 - Fully testable ChaCha20 DRNG via a userspace ChaCha20 DRNG
   implementation [3].

 - In case of using the kernel crypto API SHASH hash implementation, it
   is fully testable and tested via the NIST ACVP test framework, for
   example certificates A734, A737, and A738.

 - The LRNG offers a test interface to validate the used software hash
   implementation and in particular that the LRNG invokes the hash
   correctly, allowing a NIST ACVP-compliant test cycle - see [2]
   section 2.15.

 - Availability of stress testing covering the different code paths for
   data and mechanism (de)allocations and code paths covered with locks.

* Entropy collection

 - The LRNG is shipped with test tools allowing the collection of
   raw unconditioned entropy during runtime and boot time available at
   [1].

 - Full entropy assessment and description is provided with [2] chapter 3,
   specifically section 3.2.6.

 - Guarantee that entropy events are not credited with entropy twice
   (the existing /dev/random implementation credits HID/disk and
   interrupt events with entropy which are a derivative of each other)
   and guarantee that entropy data is not reused for two different use
   cases (as done in the existing /dev/random implementation when
   injecting a part of fast_pool into the net_rand_state).

* Configurable

 - LRNG kernel configuration allows configuration that is functionally
   equivalent to the existing /dev/random. Non-compiled additional code
   is folded into no-ops.

 - The following additional functions are compile-time selectable
   independent of each other:

  + Enabling of switchable cryptographic implementation support. This
    allows enabling an SP800-90A DRBG.

  + Enabling of using Jitter RNG noise source.

  + Enabling of noise source health tests.

  + Enabling of test interface allowing to enable each test interface
    individually.

  + Enabling of the power-up self test.

 - At boot-time, the SP800-90B health tests can be enabled as outlined
   in [2] section 2.5.4.

 - At boot-time, the entropy rate used to credit the external CPU-based
   noise source and Jitter RNG noise source can be configured including
   setting an entropy rate of zero or full entropy - see [2] sections
   2.5.2 and 2.5.3.

* Run-time pluggable cryptographic implementations used for all data
  processing steps specified in [2] section 2.2

 - The DRNG can be replaced with a different implementation allowing
   any type of DRNG to provide data via the output interfaces. The LRNG
   provides the following types of DRNG implementations:

  + ChaCha20-based software implementation that is used per default.

  + SP800-90A DRBG using accelerated cryptographic implementations that
    may sleep.

  + Any DRNG that is accessible via the kernel crypto API RNG subsystem.

 - The hash component can be replaced with any other hash implementation
   provided the implementation does not sleep. The LRNG provides the
   access to the following types of non-sleeping hash implementations:

  + SHA-256 software implementation that is used per default. Due to
    kernel build system inconsistencies, the software SHA-1 implementation
    is used if the kernel crypto API is not compiled.

  + SHA-512 hash using the fastest hash implementation available via the
    kernel crypto API SHASH subsystem.

* Code structure

 - The LRNG source code is available for current upstream Linux kernel
   separate to the existing /dev/random which means that users who are
   conservative can use the unchanged existing /dev/random implementation.

 - Back-port patches are available at [5] to apply the LRNG to Linux
   kernel versions of 5.8, 5.4, 4.19, 4.14, 4.12, and 4.10. Patches for
   other kernel versions are easily derived from the existing ones.

Booting the patch with the kernel command line option
"dyndbg=file drivers/char/lrng/* +p" generates logs indicating the
operation of the LRNG. Each log is pre-pended with "lrng".

An entropy analysis is performed on the following systems - details
are given in [2] appendix C:

* x86 KVM virtualized guest 32 and 64 bit systems

* x86 bare metal

* older and newer ARMv7 system

* ARM64

* POWER7 LE and POWER 8 BE

* IBM Z System mainframe

* old MIPS embedded device

* testing with GCC and Clang

[1] https://www.chronox.de/lrng.html - If the patch is accepted, I would
be volunteering to convert the documentation into RST format and
contribute it to the Linux kernel documentation directory.

[2] https://www.chronox.de/lrng/doc/lrng.pdf

[3] https://www.chronox.de/chacha20_drng.html

[4] https://www.bsi.bund.de/SharedDocs/Downloads/EN/BSI/Publications/Studies/LinuxRNG/LinuxRNG_EN_V4_1.pdf

[5] https://github.com/smuellerDD/lrng/tree/master/backports

Changes (compared to the previous patch set) - individual patches
are visible at https://github.com/smuellerDD/lrng/commits/master:

* fix display of available entropy - the fix only affects the display of
  available entropy at /proc/sys/kernel/random/entropy_avail

* simplify code to obtain available and max entropy

* always reset entropy gathering interface when listener detaches

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
CC: Eric Biggers <ebiggers@kernel.org>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>

Stephan Mueller (13):
  Linux Random Number Generator
  LRNG - allocate one DRNG instance per NUMA node
  LRNG - sysctls and /proc interface
  LRNG - add switchable DRNG support
  LRNG - add common generic hash support
  crypto: DRBG - externalize DRBG functions for LRNG
  LRNG - add SP800-90A DRBG extension
  LRNG - add kernel crypto API PRNG extension
  crypto: provide access to a static Jitter RNG state
  LRNG - add Jitter RNG fast noise source
  LRNG - add SP800-90B compliant health tests
  LRNG - add interface for gathering of raw entropy
  LRNG - add power-on and runtime self-tests

 MAINTAINERS                                   |   7 +
 crypto/drbg.c                                 |  16 +-
 crypto/jitterentropy-kcapi.c                  |   3 +-
 crypto/jitterentropy.c                        |  31 +-
 drivers/char/Kconfig                          |   2 +
 drivers/char/Makefile                         |   9 +-
 drivers/char/lrng/Kconfig                     | 353 +++++++++
 drivers/char/lrng/Makefile                    |  20 +
 drivers/char/lrng/lrng_archrandom.c           |  93 +++
 drivers/char/lrng/lrng_aux.c                  | 136 ++++
 drivers/char/lrng/lrng_chacha20.c             | 352 +++++++++
 drivers/char/lrng/lrng_chacha20.h             |  29 +
 drivers/char/lrng/lrng_drbg.c                 | 197 +++++
 drivers/char/lrng/lrng_drng.c                 | 406 +++++++++++
 drivers/char/lrng/lrng_health.c               | 407 +++++++++++
 drivers/char/lrng/lrng_interfaces.c           | 649 +++++++++++++++++
 drivers/char/lrng/lrng_internal.h             | 429 +++++++++++
 drivers/char/lrng/lrng_jent.c                 |  88 +++
 drivers/char/lrng/lrng_kcapi.c                | 228 ++++++
 drivers/char/lrng/lrng_kcapi_hash.c           |  97 +++
 drivers/char/lrng/lrng_kcapi_hash.h           |  19 +
 drivers/char/lrng/lrng_numa.c                 | 108 +++
 drivers/char/lrng/lrng_pool.c                 | 457 ++++++++++++
 drivers/char/lrng/lrng_proc.c                 | 182 +++++
 drivers/char/lrng/lrng_selftest.c             | 344 +++++++++
 drivers/char/lrng/lrng_sw_noise.c             | 466 ++++++++++++
 drivers/char/lrng/lrng_sw_noise.h             |  56 ++
 drivers/char/lrng/lrng_switch.c               | 203 ++++++
 drivers/char/lrng/lrng_testing.c              | 689 ++++++++++++++++++
 include/crypto/drbg.h                         |   7 +
 .../crypto/internal}/jitterentropy.h          |   3 +
 include/linux/lrng.h                          |  79 ++
 32 files changed, 6155 insertions(+), 10 deletions(-)
 create mode 100644 drivers/char/lrng/Kconfig
 create mode 100644 drivers/char/lrng/Makefile
 create mode 100644 drivers/char/lrng/lrng_archrandom.c
 create mode 100644 drivers/char/lrng/lrng_aux.c
 create mode 100644 drivers/char/lrng/lrng_chacha20.c
 create mode 100644 drivers/char/lrng/lrng_chacha20.h
 create mode 100644 drivers/char/lrng/lrng_drbg.c
 create mode 100644 drivers/char/lrng/lrng_drng.c
 create mode 100644 drivers/char/lrng/lrng_health.c
 create mode 100644 drivers/char/lrng/lrng_interfaces.c
 create mode 100644 drivers/char/lrng/lrng_internal.h
 create mode 100644 drivers/char/lrng/lrng_jent.c
 create mode 100644 drivers/char/lrng/lrng_kcapi.c
 create mode 100644 drivers/char/lrng/lrng_kcapi_hash.c
 create mode 100644 drivers/char/lrng/lrng_kcapi_hash.h
 create mode 100644 drivers/char/lrng/lrng_numa.c
 create mode 100644 drivers/char/lrng/lrng_pool.c
 create mode 100644 drivers/char/lrng/lrng_proc.c
 create mode 100644 drivers/char/lrng/lrng_selftest.c
 create mode 100644 drivers/char/lrng/lrng_sw_noise.c
 create mode 100644 drivers/char/lrng/lrng_sw_noise.h
 create mode 100644 drivers/char/lrng/lrng_switch.c
 create mode 100644 drivers/char/lrng/lrng_testing.c
 rename {crypto => include/crypto/internal}/jitterentropy.h (84%)
 create mode 100644 include/linux/lrng.h

-- 
2.26.2

[PATCH v36 01/13] Linux Random Number Generator

[Stephan Müller]

In an effort to provide a flexible implementation for a random number
generator that also delivers entropy during early boot time, allows
replacement of the deterministic random number generation mechanism,
implement the various components in separate code for easier
maintenance, and provide compliance to SP800-90[A|B|C], introduce
the Linux Random Number Generator (LRNG) framework.

The general design is as follows. Additional implementation details
are given in [1]. The LRNG consists of the following components:

1. The LRNG implements a DRNG. The DRNG always generates the
requested amount of output. When using the SP800-90A terminology
it operates without prediction resistance. The secondary DRNG
maintains a counter of how many bytes were generated since last
re-seed and a timer of the elapsed time since last re-seed. If either
the counter or the timer reaches a threshold, the secondary DRNG is
seeded from the entropy pool.

In case the Linux kernel detects a NUMA system, one secondary DRNG
instance per NUMA node is maintained.

2. The DRNG is seeded by concatenating the data from the
following sources:

(a) the output of the entropy pool,

(b) the Jitter RNG if available and enabled, and

(c) the CPU-based noise source such as Intel RDRAND if available and
enabled.

The entropy estimate of the data of all noise sources are added to
form the entropy estimate of the data used to seed the DRNG with.
The LRNG ensures, however, that the DRNG after seeding is at
maximum the security strength of the DRNG.

The LRNG is designed such that none of these noise sources can dominate
the other noise sources to provide seed data to the DRNG during due to
the following:

(a) During boot time, the amount of received interrupts are the trigger
points to (re)seed the DRNG.

(b) At runtime, the available entropy from the slow noise source is
concatenated with a pre-defined amount of data from the fast noise
sources. In addition, each DRNG reseed operation triggers external
noise source providers to deliver one block of data.

3. The entropy pool accumulates entropy obtained from certain events,
which will henceforth be collectively called "slow noise sources".
The entropy pool collects noise data from slow noise sources. Any data
received by the LRNG from the slow noise sources is inserted into a
per-CPU entropy pool using a hash operation that can be changed during
runtime. Per default, SHA-256 is used.

 (a) When an interrupt occurs, the high-resolution time stamp is mixed
into the per-CPU entropy pool. This time stamp is credited with
heuristically implied entropy.

 (b) HID event data like the key stroke or the mouse coordinates are
mixed into the per-CPU entropy pool. This data is not credited with
entropy by the LRNG.

 (c) Device drivers may provide data that is mixed into an auxiliary
pool using the same hash that is used to process the per-CPU entropy
pool. This data is not credited with entropy by the LRNG.

Any data provided from user space by either writing to /dev/random,
/dev/urandom or the IOCTL of RNDADDENTROPY on both device files
are always injected into the auxiliary pool.

In addition, when a hardware random number generator covered by the
Linux kernel HW generator framework wants to deliver random numbers,
it is injected into the auxiliary pool as well. HW generator noise source
is handled separately from the other noise source due to the fact that
the HW generator framework may decide by itself when to deliver data
whereas the other noise sources always requested for data driven by the
LRNG operation. Similarly any user space provided data is inserted into
the entropy pool.

When seed data for the DRNG is to be generated, all per-CPU
entropy pools and the auxiliary pool are hashed. The message digest
forms the new auxiliary pool state. At the same time, this data
is used for seeding the DRNG.

To speed up the interrupt handling code of the LRNG, the time stamp
collected for an interrupt event is truncated to the 8 least
significant bits. 64 truncated time stamps are concatenated and then
jointly inserted into the per-CPU entropy pool. During boot time,
until the fully seeded stage is reached, each time stamp with its
32 least significant bits is are concatenated. When 16 such events
are received, they are injected into the per-CPU entropy pool.

The LRNG allows the DRNG mechanism to be changed at runtime. Per default,
a ChaCha20-based DRNG is used. The ChaCha20-DRNG implemented for the
LRNG is also provided as a stand-alone user space deterministic random
number generator. The LRNG also offers an SP800-90A DRBG based on the
Linux kernel crypto API DRBG implementation.

The processing of entropic data from the noise source before injecting
them into the DRNG is performed with the following mathematical
operations:

1. Truncation: The received time stamps are truncated to 8 least
significant bits (or 32 least significant bits during boot time)

2. Concatenation: The received and truncated time stamps as well as
auxiliary 32 bit words are concatenated to fill the per-CPU data
array that is capable of holding 64 8-bit words.

3. Hashing: A set of concatenated time stamp data received from the
interrupts are hashed together with the current existing per-CPU
entropy pool state. The resulting message digest is the new per-CPU
entropy pool state.

4. Hashing: When new data is added to the auxiliary pool, the data
is hashed together with the auxiliary pool to form a new auxiliary
pool state.

5. Hashing: A message digest of all per-CPU entropy pools and the
auxiliary pool is calculated which forms the new auxiliary pool
state. At the same time, this message digest is used to fill the
slow noise source output buffer discussed in the following.

6. Truncation: The most-significant bits (MSB) defined by the
requested number of bits (commonly equal to the security strength
of the DRBG) or the entropy available transported with the buffer
(which is the minimum of the message digest size and the available
entropy in all entropy pools and the auxiliary pool), whatever is
smaller, are obtained from the slow noise source output buffer.

7. Concatenation: The temporary seed buffer used to seed the DRNG
is a concatenation of the slow noise source buffer, the Jitter RNG
output, the CPU noise source output, and the current time.

The DRNG always tries to seed itself with 256 bits of entropy, except
during boot. In any case, if the noise sources cannot deliver that
amount, the available entropy is used and the DRNG keeps track on how
much entropy it was seeded with. The entropy implied by the LRNG
available in the entropy pool may be too conservative. To ensure
that during boot time all available entropy from the entropy pool is
transferred to the DRNG, the hash_df function always generates 256
data bits during boot to seed the DRNG. During boot, the DRNG is
seeded as follows:

1. The DRNG is reseeded from the entropy pool and potentially the fast
noise sources if the entropy pool has collected at least 32 bits of
entropy from the interrupt noise source. The goal of this step is to
ensure that the DRNG receives some initial entropy as early as
possible. In addition it receives the entropy available from
the fast noise sources.

2. The DRNG is reseeded from the entropy pool and potentially the fast
noise sources if all noise sources collectively can provide at least
128 bits of entropy.

3. The DRNG is reseeded from the entropy pool and potentially the fast
noise sources if all noise sources collectivel can provide at least 256
bits.

At the time of the reseeding steps, the DRNG requests as much entropy as
is available in order to skip certain steps and reach the seeding level
of 256 bits. This may imply that one or more of the aforementioned steps
are skipped.

In all listed steps, the DRNG is (re)seeded with a number of random
bytes from the entropy pool that is at most the amount of entropy
present in the entropy pool. This means that when the entropy pool
contains 128 or 256 bits of entropy, the DRNG is seeded with that
amount of entropy as well.

Before the DRNG is seeded with 256 bits of entropy in step 3,
requests of random data from /dev/random and the getrandom system
call are not processed.

The hash operation providing random data from the entropy pools will
always require that all entropy sources collectively can deliver at
least 128 entropy bits.

The DRNG operates as deterministic random number generator with the
following properties:

* The maximum number of random bytes that can be generated with one
DRNG generate operation is limited to 4096 bytes. When longer random
numbers are requested, multiple DRNG generate operations are performed.
The ChaCha20 DRNG as well as the SP800-90A DRBGs implement an update of
their state after completing a generate request for backtracking
resistance.

* The secondary DRNG is reseeded with whatever entropy is available –
in the worst case where no additional entropy can be provided by the
noise sources, the DRNG is not re-seeded and continues its operation
to try to reseed again after again the expiry of one of these thresholds:

 - If the last reseeding of the secondary DRNG is more than 600 seconds
   ago, or

 - 2^20 DRNG generate operations are performed, whatever comes first, or

 - the secondary DRNG is forced to reseed before the next generation of
   random numbers if data has been injected into the LRNG by writing data
   into /dev/random or /dev/urandom.

The chosen values prevent high-volume requests from user space to cause
frequent reseeding operations which drag down the performance of the
DRNG.

With the automatic reseeding after 600 seconds, the LRNG is triggered
to reseed itself before the first request after a suspend that put the
hardware to sleep for longer than 600 seconds.

To support smaller devices including IoT environments, this patch
allows reducing the runtime memory footprint of the LRNG at compile
time by selecting smaller collection data sizes.

When selecting the compilation of a kernel for a small environment,
prevent the allocation of a buffer up to 4096 bytes to serve user space
requests. In this case, the stack variable of 64 bytes is used to serve
all user space requests.

The LRNG has the following properties:

* internal noise source: interrupts timing with fast boot time seeding

* high performance of interrupt handling code: The LRNG impact on the
interrupt handling has been reduced to a minimum. On one example
system, the LRNG interrupt handling code in its fastest configuration
executes within an average 55 cycles whereas the existing
/dev/random on the same device takes about 97 cycles when measuring
the execution time of add_interrupt_randomness().

* use of almost never contended lock for hashing operation to collect
  raw entropy supporting concurrency-free use of massive parallel
  systems - worst case rate of contention is the number of DRNG
  reseeds, usually: number of NUMA nodes contentions per 5 minutes.

* use of standalone ChaCha20 based RNG with the option to use a
  different DRNG selectable at compile time

* instantiate one DRNG per NUMA node

* support for runtime switchable output DRNGs

* use of runtime-switchable hash for conditioning implementation
following widely accepted approach

* compile-time selectable collection size

* support of small systems by allowing the reduction of the
runtime memory needs

Further details including the rationale for the design choices and
properties of the LRNG together with testing is provided at [1].
In addition, the documentation explains the conducted regression
tests to verify that the LRNG is API and ABI compatible with the
existing /dev/random implementation.

[1] https://www.chronox.de/lrng.html

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Mathematical aspects Reviewed-by: "Peter, Matthias" <matthias.peter@bsi.bund.de>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 MAINTAINERS                         |   7 +
 drivers/char/Kconfig                |   2 +
 drivers/char/Makefile               |   9 +-
 drivers/char/lrng/Kconfig           |  72 +++
 drivers/char/lrng/Makefile          |   9 +
 drivers/char/lrng/lrng_archrandom.c |  93 ++++
 drivers/char/lrng/lrng_aux.c        | 136 ++++++
 drivers/char/lrng/lrng_chacha20.c   | 352 +++++++++++++++
 drivers/char/lrng/lrng_chacha20.h   |  29 ++
 drivers/char/lrng/lrng_drng.c       | 406 +++++++++++++++++
 drivers/char/lrng/lrng_interfaces.c | 651 ++++++++++++++++++++++++++++
 drivers/char/lrng/lrng_internal.h   | 420 ++++++++++++++++++
 drivers/char/lrng/lrng_pool.c       | 457 +++++++++++++++++++
 drivers/char/lrng/lrng_sw_noise.c   | 466 ++++++++++++++++++++
 drivers/char/lrng/lrng_sw_noise.h   |  56 +++
 include/linux/lrng.h                |  79 ++++
 16 files changed, 3243 insertions(+), 1 deletion(-)
 create mode 100644 drivers/char/lrng/Kconfig
 create mode 100644 drivers/char/lrng/Makefile
 create mode 100644 drivers/char/lrng/lrng_archrandom.c
 create mode 100644 drivers/char/lrng/lrng_aux.c
 create mode 100644 drivers/char/lrng/lrng_chacha20.c
 create mode 100644 drivers/char/lrng/lrng_chacha20.h
 create mode 100644 drivers/char/lrng/lrng_drng.c
 create mode 100644 drivers/char/lrng/lrng_interfaces.c
 create mode 100644 drivers/char/lrng/lrng_internal.h
 create mode 100644 drivers/char/lrng/lrng_pool.c
 create mode 100644 drivers/char/lrng/lrng_sw_noise.c
 create mode 100644 drivers/char/lrng/lrng_sw_noise.h
 create mode 100644 include/linux/lrng.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 867157311dc8..750c9fa43cc7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10110,6 +10110,13 @@ F:	Documentation/litmus-tests/
 F:	Documentation/memory-barriers.txt
 F:	tools/memory-model/
 
+LINUX RANDOM NUMBER GENERATOR (LRNG) DRIVER
+M:	Stephan Mueller <smueller@chronox.de>
+S:	Maintained
+W:	https://www.chronox.de/lrng.html
+F:	drivers/char/lrng/*
+F:	include/linux/lrng.h
+
 LIS3LV02D ACCELEROMETER DRIVER
 M:	Eric Piel <eric.piel@tremplin-utc.net>
 S:	Maintained
diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
index b1bd336761b1..8a5b8a2db2c7 100644
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -470,6 +470,8 @@ config ADI
 	  and SSM (Silicon Secured Memory).  Intended consumers of this
 	  driver include crash and makedumpfile.
 
+source "drivers/char/lrng/Kconfig"
+
 endmenu
 
 config RANDOM_TRUST_CPU
diff --git a/drivers/char/Makefile b/drivers/char/Makefile
index ffce287ef415..2110d3e28cf2 100644
--- a/drivers/char/Makefile
+++ b/drivers/char/Makefile
@@ -3,7 +3,14 @@
 # Makefile for the kernel character device drivers.
 #
 
-obj-y				+= mem.o random.o
+obj-y				+= mem.o
+
+ifeq ($(CONFIG_LRNG),y)
+  obj-y				+= lrng/
+else
+  obj-y				+= random.o
+endif
+
 obj-$(CONFIG_TTY_PRINTK)	+= ttyprintk.o
 obj-y				+= misc.o
 obj-$(CONFIG_ATARI_DSP56K)	+= dsp56k.o
diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
new file mode 100644
index 000000000000..fbbcf2ef43b6
--- /dev/null
+++ b/drivers/char/lrng/Kconfig
@@ -0,0 +1,72 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Linux Random Number Generator configuration
+#
+
+menuconfig LRNG
+	bool "Linux Random Number Generator"
+	select CRYPTO_LIB_SHA256 if CRYPTO
+	help
+	  The Linux Random Number Generator (LRNG) is the replacement
+	  of the existing /dev/random provided with drivers/char/random.c.
+	  It generates entropy from different noise sources and
+	  delivers significant entropy during boot.
+
+if LRNG
+
+choice
+	prompt "LRNG Entropy Collection Pool Size"
+	default LRNG_COLLECTION_SIZE_64
+	help
+	  Select the size of the LRNG entropy collection pool
+	  storing data without performing a compression operation.
+	  The larger the collection size is, the faster the
+	  average interrupt handling will be. However, on the
+	  other hand the time until the LRNG received full entropy
+	  during boot time is longer because entropy is only awarded
+	  to events once they are compressed. The collection
+	  size represents the number of bytes of the per-CPU
+	  memory used to batch up entropy event data.
+
+	  The default value is good for regular operations. Choose
+	  larger sizes for servers whose boot time is of less
+	  interest. Runtime memory is precious, choose a smaller
+	  size.
+
+	  The collection size is unrelated to the entropy rate
+	  or the amount of entropy the LRNG can process.
+
+	config LRNG_COLLECTION_SIZE_16
+		bool "16 interrupt events"
+
+	config LRNG_COLLECTION_SIZE_32
+		bool "32 interrupt events"
+
+	config LRNG_COLLECTION_SIZE_64
+		bool "64 interrupt events (default)"
+
+	config LRNG_COLLECTION_SIZE_128
+		bool "128 interrupt events"
+
+	config LRNG_COLLECTION_SIZE_256
+		bool "256 interrupt events"
+
+	config LRNG_COLLECTION_SIZE_512
+		bool "512 interrupt events"
+
+	config LRNG_COLLECTION_SIZE_1024
+		bool "1024 interrupt events"
+
+endchoice
+
+config LRNG_COLLECTION_SIZE
+	int
+	default 16 if LRNG_COLLECTION_SIZE_16
+	default 32 if LRNG_COLLECTION_SIZE_32
+	default 64 if LRNG_COLLECTION_SIZE_64
+	default 128 if LRNG_COLLECTION_SIZE_128
+	default 256 if LRNG_COLLECTION_SIZE_256
+	default 512 if LRNG_COLLECTION_SIZE_512
+	default 1024 if LRNG_COLLECTION_SIZE_1024
+
+endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
new file mode 100644
index 000000000000..e72e01c15bb9
--- /dev/null
+++ b/drivers/char/lrng/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Linux Random Number Generator.
+#
+
+obj-y				+= lrng_pool.o lrng_aux.o \
+				   lrng_sw_noise.o lrng_archrandom.o \
+				   lrng_drng.o lrng_chacha20.o \
+				   lrng_interfaces.o
diff --git a/drivers/char/lrng/lrng_archrandom.c b/drivers/char/lrng/lrng_archrandom.c
new file mode 100644
index 000000000000..5e81c4e856d5
--- /dev/null
+++ b/drivers/char/lrng/lrng_archrandom.c
@@ -0,0 +1,93 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG Fast Noise Source: CPU-based noise source
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/random.h>
+
+#include "lrng_internal.h"
+
+/*
+ * Estimated entropy of data is a 32th of LRNG_DRNG_SECURITY_STRENGTH_BITS.
+ * As we have no ability to review the implementation of those noise sources,
+ * it is prudent to have a conservative estimate here.
+ */
+#define LRNG_ARCHRANDOM_DEFAULT_STRENGTH (LRNG_DRNG_SECURITY_STRENGTH_BITS>>5)
+#define LRNG_ARCHRANDOM_TRUST_CPU_STRENGTH LRNG_DRNG_SECURITY_STRENGTH_BITS
+#ifdef CONFIG_RANDOM_TRUST_CPU
+static u32 archrandom = LRNG_ARCHRANDOM_TRUST_CPU_STRENGTH;
+#else
+static u32 archrandom = LRNG_ARCHRANDOM_DEFAULT_STRENGTH;
+#endif
+module_param(archrandom, uint, 0644);
+MODULE_PARM_DESC(archrandom, "Entropy in bits of 256 data bits from CPU noise source (e.g. RDRAND)");
+
+static int __init lrng_parse_trust_cpu(char *arg)
+{
+	int ret;
+	bool trust_cpu = false;
+
+	ret = kstrtobool(arg, &trust_cpu);
+	if (ret)
+		return ret;
+
+	if (trust_cpu)
+		archrandom = LRNG_ARCHRANDOM_TRUST_CPU_STRENGTH;
+	else
+		archrandom = LRNG_ARCHRANDOM_DEFAULT_STRENGTH;
+
+	return 0;
+}
+early_param("random.trust_cpu", lrng_parse_trust_cpu);
+
+/**
+ * lrng_get_arch() - Get CPU noise source entropy
+ *
+ * @outbuf: buffer to store entropy of size LRNG_DRNG_SECURITY_STRENGTH_BYTES
+ *
+ * Return:
+ * * > 0 on success where value provides the added entropy in bits
+ * *   0 if no fast source was available
+ */
+u32 lrng_get_arch(u8 *outbuf)
+{
+	u32 i, ent_bits = archrandom;
+
+	/* operate on full blocks */
+	BUILD_BUG_ON(LRNG_DRNG_SECURITY_STRENGTH_BYTES % sizeof(unsigned long));
+	/* ensure we have aligned buffers */
+	BUILD_BUG_ON(LRNG_KCAPI_ALIGN % sizeof(unsigned long));
+
+	if (!ent_bits)
+		return 0;
+
+	for (i = 0; i < LRNG_DRNG_SECURITY_STRENGTH_BYTES;
+	     i += sizeof(unsigned long)) {
+		if (!arch_get_random_seed_long((unsigned long *)(outbuf + i)) &&
+		    !arch_get_random_long((unsigned long *)(outbuf + i))) {
+			archrandom = 0;
+			return 0;
+		}
+	}
+
+	/* Obtain entropy statement -- cap entropy to buffer size in bits */
+	ent_bits = min_t(u32, ent_bits, LRNG_DRNG_SECURITY_STRENGTH_BITS);
+	pr_debug("obtained %u bits of entropy from CPU RNG noise source\n",
+		 ent_bits);
+	return ent_bits;
+}
+
+u32 lrng_slow_noise_req_entropy(u32 required_entropy_bits)
+{
+	u32 arch_ent_bits = min_t(u32, archrandom,
+				  LRNG_DRNG_SECURITY_STRENGTH_BITS);
+	u32 fast_noise_entropy = arch_ent_bits + lrng_jent_entropylevel();
+
+	if (fast_noise_entropy > required_entropy_bits)
+		return 0;
+	return (required_entropy_bits - fast_noise_entropy);
+}
diff --git a/drivers/char/lrng/lrng_aux.c b/drivers/char/lrng/lrng_aux.c
new file mode 100644
index 000000000000..9f46c5707517
--- /dev/null
+++ b/drivers/char/lrng/lrng_aux.c
@@ -0,0 +1,136 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG auxiliary interfaces
+ *
+ * Copyright (C) 2019 Stephan Mueller <smueller@chronox.de>
+ * Copyright (C) 2017 Jason A. Donenfeld <Jason@zx2c4.com>. All
+ * Rights Reserved.
+ * Copyright (C) 2016 Jason Cooper <jason@lakedaemon.net>
+ */
+
+#include <linux/mm.h>
+#include <linux/random.h>
+
+#include "lrng_internal.h"
+
+struct batched_entropy {
+	union {
+		u64 entropy_u64[LRNG_DRNG_BLOCKSIZE / sizeof(u64)];
+		u32 entropy_u32[LRNG_DRNG_BLOCKSIZE / sizeof(u32)];
+	};
+	unsigned int position;
+	spinlock_t batch_lock;
+};
+
+/*
+ * Get a random word for internal kernel use only. The quality of the random
+ * number is as good as /dev/urandom, but there is no backtrack protection,
+ * with the goal of being quite fast and not depleting entropy.
+ */
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = {
+	.batch_lock	= __SPIN_LOCK_UNLOCKED(batched_entropy_u64.lock),
+};
+
+u64 get_random_u64(void)
+{
+	u64 ret;
+	unsigned long flags;
+	struct batched_entropy *batch;
+
+	lrng_debug_report_seedlevel("get_random_u64");
+
+	batch = raw_cpu_ptr(&batched_entropy_u64);
+	spin_lock_irqsave(&batch->batch_lock, flags);
+	if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0) {
+		lrng_drng_get_atomic((u8 *)batch->entropy_u64,
+				      LRNG_DRNG_BLOCKSIZE);
+		batch->position = 0;
+	}
+	ret = batch->entropy_u64[batch->position++];
+	spin_unlock_irqrestore(&batch->batch_lock, flags);
+	return ret;
+}
+EXPORT_SYMBOL(get_random_u64);
+
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32) = {
+	.batch_lock	= __SPIN_LOCK_UNLOCKED(batched_entropy_u32.lock),
+};
+
+u32 get_random_u32(void)
+{
+	u32 ret;
+	unsigned long flags;
+	struct batched_entropy *batch;
+
+	lrng_debug_report_seedlevel("get_random_u32");
+
+	batch = raw_cpu_ptr(&batched_entropy_u32);
+	spin_lock_irqsave(&batch->batch_lock, flags);
+	if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0) {
+		lrng_drng_get_atomic((u8 *)batch->entropy_u32,
+				      LRNG_DRNG_BLOCKSIZE);
+		batch->position = 0;
+	}
+	ret = batch->entropy_u32[batch->position++];
+	spin_unlock_irqrestore(&batch->batch_lock, flags);
+	return ret;
+}
+EXPORT_SYMBOL(get_random_u32);
+
+/*
+ * It's important to invalidate all potential batched entropy that might
+ * be stored before the crng is initialized, which we can do lazily by
+ * simply resetting the counter to zero so that it's re-extracted on the
+ * next usage.
+ */
+void invalidate_batched_entropy(void)
+{
+	int cpu;
+	unsigned long flags;
+
+	for_each_possible_cpu(cpu) {
+		struct batched_entropy *batched_entropy;
+
+		batched_entropy = per_cpu_ptr(&batched_entropy_u32, cpu);
+		spin_lock_irqsave(&batched_entropy->batch_lock, flags);
+		batched_entropy->position = 0;
+		spin_unlock(&batched_entropy->batch_lock);
+
+		batched_entropy = per_cpu_ptr(&batched_entropy_u64, cpu);
+		spin_lock(&batched_entropy->batch_lock);
+		batched_entropy->position = 0;
+		spin_unlock_irqrestore(&batched_entropy->batch_lock, flags);
+	}
+}
+
+/**
+ * randomize_page - Generate a random, page aligned address
+ * @start:	The smallest acceptable address the caller will take.
+ * @range:	The size of the area, starting at @start, within which the
+ *		random address must fall.
+ *
+ * If @start + @range would overflow, @range is capped.
+ *
+ * NOTE: Historical use of randomize_range, which this replaces, presumed that
+ * @start was already page aligned.  We now align it regardless.
+ *
+ * Return: A page aligned address within [start, start + range).  On error,
+ * @start is returned.
+ */
+unsigned long randomize_page(unsigned long start, unsigned long range)
+{
+	if (!PAGE_ALIGNED(start)) {
+		range -= PAGE_ALIGN(start) - start;
+		start = PAGE_ALIGN(start);
+	}
+
+	if (start > ULONG_MAX - range)
+		range = ULONG_MAX - start;
+
+	range >>= PAGE_SHIFT;
+
+	if (range == 0)
+		return start;
+
+	return start + (get_random_long() % range << PAGE_SHIFT);
+}
diff --git a/drivers/char/lrng/lrng_chacha20.c b/drivers/char/lrng/lrng_chacha20.c
new file mode 100644
index 000000000000..1019e0024676
--- /dev/null
+++ b/drivers/char/lrng/lrng_chacha20.c
@@ -0,0 +1,352 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Backend for the LRNG providing the cryptographic primitives using
+ * ChaCha20 cipher implementations.
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <crypto/chacha.h>
+#include <crypto/sha.h>
+#include <linux/lrng.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+
+#include "lrng_chacha20.h"
+#include "lrng_internal.h"
+
+/******************************* ChaCha20 DRNG *******************************/
+
+#define CHACHA_BLOCK_WORDS	(CHACHA_BLOCK_SIZE / sizeof(u32))
+
+struct chacha20_state {
+	struct chacha20_block block;
+};
+
+/*
+ * Have a static memory blocks for the ChaCha20 DRNG instance to avoid calling
+ * kmalloc too early in the boot cycle. For subsequent allocation requests,
+ * such as per-NUMA-node DRNG instances, kmalloc will be used.
+ */
+struct chacha20_state chacha20 __latent_entropy;
+
+/**
+ * Update of the ChaCha20 state by either using an unused buffer part or by
+ * generating one ChaCha20 block which is half of the state of the ChaCha20.
+ * The block is XORed into the key part of the state. This shall ensure
+ * backtracking resistance as well as a proper mix of the ChaCha20 state once
+ * the key is injected.
+ */
+static void lrng_chacha20_update(struct chacha20_state *chacha20_state,
+				 __le32 *buf, u32 used_words)
+{
+	struct chacha20_block *chacha20 = &chacha20_state->block;
+	u32 i;
+	__le32 tmp[CHACHA_BLOCK_WORDS];
+
+	BUILD_BUG_ON(sizeof(struct chacha20_block) != CHACHA_BLOCK_SIZE);
+	BUILD_BUG_ON(CHACHA_BLOCK_SIZE != 2 * CHACHA_KEY_SIZE);
+
+	if (used_words > CHACHA_KEY_SIZE_WORDS) {
+		chacha20_block(&chacha20->constants[0], (u8 *)tmp);
+		for (i = 0; i < CHACHA_KEY_SIZE_WORDS; i++)
+			chacha20->key.u[i] ^= le32_to_cpu(tmp[i]);
+		memzero_explicit(tmp, sizeof(tmp));
+	} else {
+		for (i = 0; i < CHACHA_KEY_SIZE_WORDS; i++)
+			chacha20->key.u[i] ^= le32_to_cpu(buf[i + used_words]);
+	}
+
+	/* Deterministic increment of nonce as required in RFC 7539 chapter 4 */
+	chacha20->nonce[0]++;
+	if (chacha20->nonce[0] == 0)
+		chacha20->nonce[1]++;
+	if (chacha20->nonce[1] == 0)
+		chacha20->nonce[2]++;
+
+	/* Leave counter untouched as it is start value is undefined in RFC */
+}
+
+/*
+ * Seed the ChaCha20 DRNG by injecting the input data into the key part of
+ * the ChaCha20 state. If the input data is longer than the ChaCha20 key size,
+ * perform a ChaCha20 operation after processing of key size input data.
+ * This operation shall spread out the entropy into the ChaCha20 state before
+ * new entropy is injected into the key part.
+ */
+static int lrng_cc20_drng_seed_helper(void *drng, const u8 *inbuf, u32 inbuflen)
+{
+	struct chacha20_state *chacha20_state = (struct chacha20_state *)drng;
+	struct chacha20_block *chacha20 = &chacha20_state->block;
+
+	while (inbuflen) {
+		u32 i, todo = min_t(u32, inbuflen, CHACHA_KEY_SIZE);
+
+		for (i = 0; i < todo; i++)
+			chacha20->key.b[i] ^= inbuf[i];
+
+		/* Break potential dependencies between the inbuf key blocks */
+		lrng_chacha20_update(chacha20_state, NULL,
+				     CHACHA_BLOCK_WORDS);
+		inbuf += todo;
+		inbuflen -= todo;
+	}
+
+	return 0;
+}
+
+/*
+ * Chacha20 DRNG generation of random numbers: the stream output of ChaCha20
+ * is the random number. After the completion of the generation of the
+ * stream, the entire ChaCha20 state is updated.
+ *
+ * Note, as the ChaCha20 implements a 32 bit counter, we must ensure
+ * that this function is only invoked for at most 2^32 - 1 ChaCha20 blocks
+ * before a reseed or an update happens. This is ensured by the variable
+ * outbuflen which is a 32 bit integer defining the number of bytes to be
+ * generated by the ChaCha20 DRNG. At the end of this function, an update
+ * operation is invoked which implies that the 32 bit counter will never be
+ * overflown in this implementation.
+ */
+static int lrng_cc20_drng_generate_helper(void *drng, u8 *outbuf, u32 outbuflen)
+{
+	struct chacha20_state *chacha20_state = (struct chacha20_state *)drng;
+	struct chacha20_block *chacha20 = &chacha20_state->block;
+	__le32 aligned_buf[CHACHA_BLOCK_WORDS];
+	u32 ret = outbuflen, used = CHACHA_BLOCK_WORDS;
+	int zeroize_buf = 0;
+
+	while (outbuflen >= CHACHA_BLOCK_SIZE) {
+		chacha20_block(&chacha20->constants[0], outbuf);
+		outbuf += CHACHA_BLOCK_SIZE;
+		outbuflen -= CHACHA_BLOCK_SIZE;
+	}
+
+	if (outbuflen) {
+		chacha20_block(&chacha20->constants[0], (u8 *)aligned_buf);
+		memcpy(outbuf, aligned_buf, outbuflen);
+		used = ((outbuflen + sizeof(aligned_buf[0]) - 1) /
+			sizeof(aligned_buf[0]));
+		zeroize_buf = 1;
+	}
+
+	lrng_chacha20_update(chacha20_state, aligned_buf, used);
+
+	if (zeroize_buf)
+		memzero_explicit(aligned_buf, sizeof(aligned_buf));
+
+	return ret;
+}
+
+void lrng_cc20_init_state(struct chacha20_state *state)
+{
+	struct chacha20_block *chacha20 = &state->block;
+	unsigned long v;
+	u32 i;
+
+	lrng_cc20_init_rfc7539(chacha20);
+
+	for (i = 0; i < CHACHA_KEY_SIZE_WORDS; i++) {
+		chacha20->key.u[i] ^= jiffies;
+		chacha20->key.u[i] ^= random_get_entropy();
+		if (arch_get_random_seed_long(&v) || arch_get_random_long(&v))
+			chacha20->key.u[i] ^= v;
+	}
+
+	for (i = 0; i < 3; i++) {
+		chacha20->nonce[i] ^= jiffies;
+		chacha20->nonce[i] ^= random_get_entropy();
+		if (arch_get_random_seed_long(&v) || arch_get_random_long(&v))
+			chacha20->nonce[i] ^= v;
+	}
+
+	lrng_chacha20_update(state, NULL, CHACHA_BLOCK_WORDS);
+	pr_info("ChaCha20 core initialized\n");
+}
+
+void __init lrng_cc20_init_state_boot(struct chacha20_state *state)
+{
+	struct chacha20_block *chacha20 = &state->block;
+	unsigned long v;
+	u32 i;
+
+	for (i = 0; i < CHACHA_KEY_SIZE_WORDS; i++) {
+		if (arch_get_random_seed_long_early(&v) ||
+		    arch_get_random_long_early(&v))
+			chacha20->key.u[i] ^= v;
+	}
+
+	for (i = 0; i < 3; i++) {
+		if (arch_get_random_seed_long_early(&v) ||
+		    arch_get_random_long_early(&v))
+			chacha20->nonce[i] ^= v;
+	}
+
+	lrng_chacha20_update(state, NULL, CHACHA_BLOCK_WORDS);
+}
+
+/*
+ * Allocation of the DRNG state
+ */
+static void *lrng_cc20_drng_alloc(u32 sec_strength)
+{
+	struct chacha20_state *state = NULL;
+
+	if (sec_strength > CHACHA_KEY_SIZE) {
+		pr_err("Security strength of ChaCha20 DRNG (%u bits) lower than requested by LRNG (%u bits)\n",
+			CHACHA_KEY_SIZE * 8, sec_strength * 8);
+		return ERR_PTR(-EINVAL);
+	}
+	if (sec_strength < CHACHA_KEY_SIZE)
+		pr_warn("Security strength of ChaCha20 DRNG (%u bits) higher than requested by LRNG (%u bits)\n",
+			CHACHA_KEY_SIZE * 8, sec_strength * 8);
+
+	state = kmalloc(sizeof(struct chacha20_state), GFP_KERNEL);
+	if (!state)
+		return ERR_PTR(-ENOMEM);
+	pr_debug("memory for ChaCha20 core allocated\n");
+
+	lrng_cc20_init_state(state);
+
+	return state;
+}
+
+static void lrng_cc20_drng_dealloc(void *drng)
+{
+	struct chacha20_state *chacha20_state = (struct chacha20_state *)drng;
+
+	if (drng == &chacha20) {
+		memzero_explicit(chacha20_state, sizeof(*chacha20_state));
+		pr_debug("static ChaCha20 core zeroized\n");
+		return;
+	}
+
+	pr_debug("ChaCha20 core zeroized and freed\n");
+	kfree_sensitive(chacha20_state);
+}
+
+/******************************* Hash Operation *******************************/
+
+#ifdef CONFIG_CRYPTO_LIB_SHA256
+
+static u32 lrng_cc20_hash_digestsize(void *hash)
+{
+	return SHA256_DIGEST_SIZE;
+}
+
+static int lrng_cc20_hash_init(struct shash_desc *shash, void *hash)
+{
+	/*
+	 * We do not need a TFM - we only need sufficient space for
+	 * struct sha1_state on the stack.
+	 */
+	sha256_init(shash_desc_ctx(shash));
+	return 0;
+}
+
+static int lrng_cc20_hash_update(struct shash_desc *shash,
+				 const u8 *inbuf, u32 inbuflen)
+{
+	sha256_update(shash_desc_ctx(shash), inbuf, inbuflen);
+	return 0;
+}
+
+static int lrng_cc20_hash_final(struct shash_desc *shash, u8 *digest)
+{
+	sha256_final(shash_desc_ctx(shash), digest);
+	return 0;
+}
+
+static const char *lrng_cc20_hash_name(void)
+{
+	const char *cc20_hash_name = "SHA-256";
+	return cc20_hash_name;
+}
+
+#else /* CONFIG_CRYPTO_LIB_SHA256 */
+
+#include <crypto/sha1_base.h>
+
+/*
+ * If the SHA-256 support is not compiled, we fall back to SHA-1 that is always
+ * compiled and present in the kernel.
+ */
+static u32 lrng_cc20_hash_digestsize(void *hash)
+{
+	return SHA1_DIGEST_SIZE;
+}
+
+static void lrng_sha1_block_fn(struct sha1_state *sctx, const u8 *src,
+			       int blocks)
+{
+	u32 temp[SHA1_WORKSPACE_WORDS];
+
+	while (blocks--) {
+		sha1_transform(sctx->state, src, temp);
+		src += SHA1_BLOCK_SIZE;
+	}
+	memzero_explicit(temp, sizeof(temp));
+}
+
+static int lrng_cc20_hash_init(struct shash_desc *shash, void *hash)
+{
+	/*
+	 * We do not need a TFM - we only need sufficient space for
+	 * struct sha1_state on the stack.
+	 */
+	sha1_base_init(shash);
+	return 0;
+}
+
+static int lrng_cc20_hash_update(struct shash_desc *shash,
+				 const u8 *inbuf, u32 inbuflen)
+{
+	return sha1_base_do_update(shash, inbuf, inbuflen, lrng_sha1_block_fn);
+}
+
+static int lrng_cc20_hash_final(struct shash_desc *shash, u8 *digest)
+{
+	return sha1_base_do_finalize(shash, lrng_sha1_block_fn) ?:
+	       sha1_base_finish(shash, digest);
+}
+
+static const char *lrng_cc20_hash_name(void)
+{
+	const char *cc20_hash_name = "SHA-1";
+	return cc20_hash_name;
+}
+
+#endif /* CONFIG_CRYPTO_LIB_SHA256 */
+
+static void *lrng_cc20_hash_alloc(void)
+{
+	pr_info("Hash %s allocated\n", lrng_cc20_hash_name());
+	return NULL;
+}
+
+static void lrng_cc20_hash_dealloc(void *hash)
+{
+}
+
+static const char *lrng_cc20_drng_name(void)
+{
+	const char *cc20_drng_name = "ChaCha20 DRNG";
+	return cc20_drng_name;
+}
+
+const struct lrng_crypto_cb lrng_cc20_crypto_cb = {
+	.lrng_drng_name			= lrng_cc20_drng_name,
+	.lrng_hash_name			= lrng_cc20_hash_name,
+	.lrng_drng_alloc		= lrng_cc20_drng_alloc,
+	.lrng_drng_dealloc		= lrng_cc20_drng_dealloc,
+	.lrng_drng_seed_helper		= lrng_cc20_drng_seed_helper,
+	.lrng_drng_generate_helper	= lrng_cc20_drng_generate_helper,
+	.lrng_hash_alloc		= lrng_cc20_hash_alloc,
+	.lrng_hash_dealloc		= lrng_cc20_hash_dealloc,
+	.lrng_hash_digestsize		= lrng_cc20_hash_digestsize,
+	.lrng_hash_init			= lrng_cc20_hash_init,
+	.lrng_hash_update		= lrng_cc20_hash_update,
+	.lrng_hash_final		= lrng_cc20_hash_final,
+};
diff --git a/drivers/char/lrng/lrng_chacha20.h b/drivers/char/lrng/lrng_chacha20.h
new file mode 100644
index 000000000000..2340b12a56a4
--- /dev/null
+++ b/drivers/char/lrng/lrng_chacha20.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * LRNG ChaCha20 definitions
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#include <crypto/chacha.h>
+
+/* State according to RFC 7539 section 2.3 */
+struct chacha20_block {
+	u32 constants[4];
+	union {
+#define CHACHA_KEY_SIZE_WORDS (CHACHA_KEY_SIZE / sizeof(u32))
+		u32 u[CHACHA_KEY_SIZE_WORDS];
+		u8  b[CHACHA_KEY_SIZE];
+	} key;
+	u32 counter;
+	u32 nonce[3];
+};
+
+static inline void lrng_cc20_init_rfc7539(struct chacha20_block *chacha20)
+{
+	/* String "expand 32-byte k" */
+	chacha20->constants[0] = 0x61707865;
+	chacha20->constants[1] = 0x3320646e;
+	chacha20->constants[2] = 0x79622d32;
+	chacha20->constants[3] = 0x6b206574;
+}
diff --git a/drivers/char/lrng/lrng_drng.c b/drivers/char/lrng/lrng_drng.c
new file mode 100644
index 000000000000..0c4e4893297e
--- /dev/null
+++ b/drivers/char/lrng/lrng_drng.c
@@ -0,0 +1,406 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG DRNG processing
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/lrng.h>
+
+#include "lrng_internal.h"
+
+/*
+ * Maximum number of seconds between DRNG reseed intervals of the DRNG. Note,
+ * this is enforced with the next request of random numbers from the
+ * DRNG. Setting this value to zero implies a reseeding attempt before every
+ * generated random number.
+ */
+int lrng_drng_reseed_max_time = 600;
+
+static atomic_t lrng_avail = ATOMIC_INIT(0);
+
+DEFINE_MUTEX(lrng_crypto_cb_update);
+
+/* DRNG for /dev/urandom, getrandom(2), get_random_bytes */
+static struct lrng_drng lrng_drng_init = {
+	.drng		= &chacha20,
+	.crypto_cb	= &lrng_cc20_crypto_cb,
+	.lock		= __MUTEX_INITIALIZER(lrng_drng_init.lock),
+	.spin_lock	= __SPIN_LOCK_UNLOCKED(lrng_drng_init.spin_lock),
+	.hash_lock	= __RW_LOCK_UNLOCKED(lrng_drng_init.hash_lock)
+};
+
+/*
+ * DRNG for get_random_bytes when called in atomic context. This
+ * DRNG will always use the ChaCha20 DRNG. It will never benefit from a
+ * DRNG switch like the "regular" DRNG. If there was no DRNG switch, the atomic
+ * DRNG is identical to the "regular" DRNG.
+ *
+ * The reason for having this is due to the fact that DRNGs other than
+ * the ChaCha20 DRNG may sleep.
+ */
+static struct lrng_drng lrng_drng_atomic = {
+	.drng		= &chacha20,
+	.crypto_cb	= &lrng_cc20_crypto_cb,
+	.spin_lock	= __SPIN_LOCK_UNLOCKED(lrng_drng_atomic.spin_lock),
+	.hash_lock	= __RW_LOCK_UNLOCKED(lrng_drng_atomic.hash_lock)
+};
+
+/********************************** Helper ************************************/
+
+bool lrng_get_available(void)
+{
+	return likely(atomic_read(&lrng_avail));
+}
+
+void lrng_set_available(void)
+{
+	atomic_set(&lrng_avail, 1);
+}
+
+struct lrng_drng *lrng_drng_init_instance(void)
+{
+	return &lrng_drng_init;
+}
+
+struct lrng_drng *lrng_drng_atomic_instance(void)
+{
+	return &lrng_drng_atomic;
+}
+
+void lrng_drng_reset(struct lrng_drng *drng)
+{
+	atomic_set(&drng->requests, LRNG_DRNG_RESEED_THRESH);
+	drng->last_seeded = jiffies;
+	drng->fully_seeded = false;
+	drng->force_reseed = true;
+	pr_debug("reset DRNG\n");
+}
+
+/* Initialize the default DRNG during boot */
+static void lrng_drngs_init_cc20(void)
+{
+	unsigned long flags = 0;
+
+	if (lrng_get_available())
+		return;
+
+	lrng_drng_lock(&lrng_drng_init, &flags);
+	if (lrng_get_available()) {
+		lrng_drng_unlock(&lrng_drng_init, &flags);
+		return;
+	}
+
+	lrng_drng_reset(&lrng_drng_init);
+	lrng_cc20_init_state(&chacha20);
+	lrng_state_init_seed_work();
+	lrng_drng_unlock(&lrng_drng_init, &flags);
+
+	lrng_drng_lock(&lrng_drng_atomic, &flags);
+	lrng_drng_reset(&lrng_drng_atomic);
+	/*
+	 * We do not initialize the state of the atomic DRNG as it is identical
+	 * to the DRNG at this point.
+	 */
+	lrng_drng_unlock(&lrng_drng_atomic, &flags);
+
+	lrng_set_available();
+}
+
+/************************* Random Number Generation ***************************/
+
+/* Inject a data buffer into the DRNG */
+static void lrng_drng_inject(struct lrng_drng *drng,
+			     const u8 *inbuf, u32 inbuflen)
+{
+	const char *drng_type = unlikely(drng == &lrng_drng_atomic) ?
+				"atomic" : "regular";
+	unsigned long flags = 0;
+
+	BUILD_BUG_ON(LRNG_DRNG_RESEED_THRESH > INT_MAX);
+	pr_debug("seeding %s DRNG with %u bytes\n", drng_type, inbuflen);
+	lrng_drng_lock(drng, &flags);
+	if (drng->crypto_cb->lrng_drng_seed_helper(drng->drng,
+						   inbuf, inbuflen) < 0) {
+		pr_warn("seeding of %s DRNG failed\n", drng_type);
+		atomic_set(&drng->requests, 1);
+	} else {
+		pr_debug("%s DRNG stats since last seeding: %lu secs; generate calls: %d\n",
+			 drng_type,
+			 (time_after(jiffies, drng->last_seeded) ?
+			  (jiffies - drng->last_seeded) : 0) / HZ,
+			 (LRNG_DRNG_RESEED_THRESH -
+			  atomic_read(&drng->requests)));
+		drng->last_seeded = jiffies;
+		atomic_set(&drng->requests, LRNG_DRNG_RESEED_THRESH);
+		drng->force_reseed = false;
+
+		if (drng->drng == lrng_drng_atomic.drng) {
+			lrng_drng_atomic.last_seeded = jiffies;
+			atomic_set(&lrng_drng_atomic.requests,
+				   LRNG_DRNG_RESEED_THRESH);
+			lrng_drng_atomic.force_reseed = false;
+		}
+	}
+	lrng_drng_unlock(drng, &flags);
+}
+
+/*
+ * Perform the seeding of the DRNG with data from noise source
+ */
+static inline int _lrng_drng_seed(struct lrng_drng *drng)
+{
+	struct entropy_buf seedbuf __aligned(LRNG_KCAPI_ALIGN);
+	u32 total_entropy_bits;
+	int ret;
+
+	total_entropy_bits = lrng_fill_seed_buffer(drng, &seedbuf);
+
+	/* Allow the seeding operation to be called again */
+	lrng_pool_unlock();
+	lrng_init_ops(total_entropy_bits);
+	ret = total_entropy_bits >> 3;
+
+	lrng_drng_inject(drng, (u8 *)&seedbuf, sizeof(seedbuf));
+	memzero_explicit(&seedbuf, sizeof(seedbuf));
+
+	if (ret >= (int)(lrng_security_strength() >> 3))
+		drng->fully_seeded = true;
+
+	return ret;
+}
+
+static int lrng_drng_get(struct lrng_drng *drng, u8 *outbuf, u32 outbuflen);
+static void lrng_drng_seed(struct lrng_drng *drng)
+{
+	int ret = _lrng_drng_seed(drng);
+
+	BUILD_BUG_ON(LRNG_MIN_SEED_ENTROPY_BITS >
+		     LRNG_DRNG_SECURITY_STRENGTH_BITS);
+
+	/*
+	 * Reseed atomic DRNG from current DRNG,
+	 *
+	 * We can obtain random numbers from DRNG as the lock type
+	 * chosen by lrng_drng_get is usable with the current caller.
+	 */
+	if ((drng->drng != lrng_drng_atomic.drng) &&
+	    (lrng_drng_atomic.force_reseed ||
+	     atomic_read(&lrng_drng_atomic.requests) <= 0 ||
+	     time_after(jiffies, lrng_drng_atomic.last_seeded +
+			lrng_drng_reseed_max_time * HZ))) {
+		u8 seedbuf[LRNG_DRNG_SECURITY_STRENGTH_BYTES]
+						__aligned(LRNG_KCAPI_ALIGN);
+
+		ret = lrng_drng_get(drng, seedbuf, sizeof(seedbuf));
+
+		if (ret < 0) {
+			pr_warn("Error generating random numbers for atomic DRNG: %d\n",
+				ret);
+		} else {
+			lrng_drng_inject(&lrng_drng_atomic, seedbuf, ret);
+		}
+		memzero_explicit(&seedbuf, sizeof(seedbuf));
+	}
+}
+
+static inline void _lrng_drng_seed_work(struct lrng_drng *drng, u32 node)
+{
+	pr_debug("reseed triggered by interrupt noise source for DRNG on NUMA node %d\n",
+		 node);
+	lrng_drng_seed(drng);
+	if (drng->fully_seeded) {
+		/* Prevent reseed storm */
+		drng->last_seeded += node * 100 * HZ;
+		/* Prevent draining of pool on idle systems */
+		lrng_drng_reseed_max_time += 100;
+	}
+}
+
+/*
+ * DRNG reseed trigger: Kernel thread handler triggered by the schedule_work()
+ */
+void lrng_drng_seed_work(struct work_struct *dummy)
+{
+	struct lrng_drng **lrng_drng = lrng_drng_instances();
+	u32 node;
+
+	if (lrng_drng) {
+		for_each_online_node(node) {
+			struct lrng_drng *drng = lrng_drng[node];
+
+			if (drng && !drng->fully_seeded) {
+				_lrng_drng_seed_work(drng, node);
+				goto out;
+			}
+		}
+		lrng_pool_all_numa_nodes_seeded();
+	} else {
+		if (!lrng_drng_init.fully_seeded)
+			_lrng_drng_seed_work(&lrng_drng_init, 0);
+	}
+
+out:
+	/* Allow the seeding operation to be called again */
+	lrng_pool_unlock();
+}
+
+/* Force all DRNGs to reseed before next generation */
+void lrng_drng_force_reseed(void)
+{
+	struct lrng_drng **lrng_drng = lrng_drng_instances();
+	u32 node;
+
+	if (!lrng_drng) {
+		lrng_drng_init.force_reseed = true;
+		pr_debug("force reseed of initial DRNG\n");
+		return;
+	}
+	for_each_online_node(node) {
+		struct lrng_drng *drng = lrng_drng[node];
+
+		if (!drng)
+			continue;
+
+		drng->force_reseed = true;
+		pr_debug("force reseed of DRNG on node %u\n", node);
+	}
+	lrng_drng_atomic.force_reseed = true;
+}
+
+/**
+ * lrng_drng_get() - Get random data out of the DRNG which is reseeded
+ * frequently.
+ *
+ * @outbuf: buffer for storing random data
+ * @outbuflen: length of outbuf
+ *
+ * Return:
+ * * < 0 in error case (DRNG generation or update failed)
+ * * >=0 returning the returned number of bytes
+ */
+static int lrng_drng_get(struct lrng_drng *drng, u8 *outbuf, u32 outbuflen)
+{
+	unsigned long flags = 0;
+	u32 processed = 0;
+
+	if (!outbuf || !outbuflen)
+		return 0;
+
+	outbuflen = min_t(size_t, outbuflen, INT_MAX);
+
+	lrng_drngs_init_cc20();
+
+	while (outbuflen) {
+		u32 todo = min_t(u32, outbuflen, LRNG_DRNG_MAX_REQSIZE);
+		int ret;
+
+		/* All but the atomic DRNG are seeded during generation */
+		if (atomic_dec_and_test(&drng->requests) ||
+		    drng->force_reseed ||
+		    time_after(jiffies, drng->last_seeded +
+			       lrng_drng_reseed_max_time * HZ)) {
+			if (likely(drng != &lrng_drng_atomic)) {
+				if (lrng_pool_trylock())
+					atomic_set(&drng->requests, 1);
+				else
+					lrng_drng_seed(drng);
+			}
+		}
+
+		lrng_drng_lock(drng, &flags);
+		ret = drng->crypto_cb->lrng_drng_generate_helper(
+					drng->drng, outbuf + processed, todo);
+		lrng_drng_unlock(drng, &flags);
+		if (ret <= 0) {
+			pr_warn("getting random data from DRNG failed (%d)\n",
+				ret);
+			return -EFAULT;
+		}
+		processed += ret;
+		outbuflen -= ret;
+	}
+
+	return processed;
+}
+
+int lrng_drng_get_atomic(u8 *outbuf, u32 outbuflen)
+{
+	return lrng_drng_get(&lrng_drng_atomic, outbuf, outbuflen);
+}
+
+int lrng_drng_get_sleep(u8 *outbuf, u32 outbuflen)
+{
+	struct lrng_drng **lrng_drng = lrng_drng_instances();
+	struct lrng_drng *drng = &lrng_drng_init;
+	int node = numa_node_id();
+
+	might_sleep();
+
+	if (lrng_drng && lrng_drng[node] && lrng_drng[node]->fully_seeded)
+		drng = lrng_drng[node];
+
+	return lrng_drng_get(drng, outbuf, outbuflen);
+}
+
+/* Reset LRNG such that all existing entropy is gone */
+static void _lrng_reset(struct work_struct *work)
+{
+	struct lrng_drng **lrng_drng = lrng_drng_instances();
+	unsigned long flags = 0;
+
+	if (!lrng_drng) {
+		lrng_drng_lock(&lrng_drng_init, &flags);
+		lrng_drng_reset(&lrng_drng_init);
+		lrng_drng_unlock(&lrng_drng_init, &flags);
+	} else {
+		u32 node;
+
+		for_each_online_node(node) {
+			struct lrng_drng *drng = lrng_drng[node];
+
+			if (!drng)
+				continue;
+			lrng_drng_lock(drng, &flags);
+			lrng_drng_reset(drng);
+			lrng_drng_unlock(drng, &flags);
+		}
+	}
+	lrng_set_entropy_thresh(LRNG_INIT_ENTROPY_BITS);
+
+	lrng_reset_state();
+}
+
+static DECLARE_WORK(lrng_reset_work, _lrng_reset);
+
+void lrng_reset(void)
+{
+	schedule_work(&lrng_reset_work);
+}
+
+/***************************** Initialize LRNG *******************************/
+
+void __init lrng_drng_init_early(void)
+{
+	unsigned long flags = 0;
+
+	lrng_drng_lock(&lrng_drng_init, &flags);
+	lrng_cc20_init_state_boot(&chacha20);
+	lrng_drng_unlock(&lrng_drng_init, &flags);
+}
+
+static int __init lrng_init(void)
+{
+	lrng_drngs_init_cc20();
+
+	lrng_drngs_numa_alloc();
+	return 0;
+}
+
+late_initcall(lrng_init);
+
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_AUTHOR("Stephan Mueller <smueller@chronox.de>");
+MODULE_DESCRIPTION("Linux Random Number Generator");
diff --git a/drivers/char/lrng/lrng_interfaces.c b/drivers/char/lrng/lrng_interfaces.c
new file mode 100644
index 000000000000..19d01d3f7492
--- /dev/null
+++ b/drivers/char/lrng/lrng_interfaces.c
@@ -0,0 +1,651 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG User and kernel space interfaces
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/freezer.h>
+#include <linux/fs.h>
+#include <linux/genhd.h>
+#include <linux/hw_random.h>
+#include <linux/kthread.h>
+#include <linux/poll.h>
+#include <linux/preempt.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+#include <linux/syscalls.h>
+#include <linux/timex.h>
+
+#define CREATE_TRACE_POINTS
+#include <trace/events/random.h>
+
+#include "lrng_internal.h"
+
+/*
+ * If the entropy count falls under this number of bits, then we
+ * should wake up processes which are selecting or polling on write
+ * access to /dev/random.
+ */
+u32 lrng_write_wakeup_bits = LRNG_WRITE_WAKEUP_ENTROPY;
+
+static LIST_HEAD(lrng_ready_list);
+static DEFINE_SPINLOCK(lrng_ready_list_lock);
+
+static DECLARE_WAIT_QUEUE_HEAD(lrng_write_wait);
+static DECLARE_WAIT_QUEUE_HEAD(lrng_init_wait);
+static struct fasync_struct *fasync;
+
+struct ctl_table random_table[];
+
+/********************************** Helper ***********************************/
+
+/* Is the DRNG seed level too low? */
+static inline bool lrng_need_entropy(void)
+{
+	return (lrng_avail_aux_entropy() < lrng_write_wakeup_bits);
+}
+
+void lrng_writer_wakeup(void)
+{
+	if (lrng_need_entropy() && wq_has_sleeper(&lrng_write_wait)) {
+		wake_up_interruptible(&lrng_write_wait);
+		kill_fasync(&fasync, SIGIO, POLL_OUT);
+	}
+}
+
+void lrng_init_wakeup(void)
+{
+	wake_up_all(&lrng_init_wait);
+	kill_fasync(&fasync, SIGIO, POLL_IN);
+}
+
+/**
+ * lrng_process_ready_list() - Ping all kernel internal callers waiting until
+ * the DRNG is at least minimally seeded to inform that the DRNG reached that
+ * seed level.
+ *
+ * When the SP800-90B testing is enabled, the ping only happens if the SP800-90B
+ * startup health tests are completed. This implies that kernel internal
+ * callers always have an SP800-90B compliant noise source when being
+ * pinged.
+ */
+void lrng_process_ready_list(void)
+{
+	unsigned long flags;
+	struct random_ready_callback *rdy, *tmp;
+
+	if (!lrng_sp80090b_startup_complete())
+		return;
+
+	spin_lock_irqsave(&lrng_ready_list_lock, flags);
+	list_for_each_entry_safe(rdy, tmp, &lrng_ready_list, list) {
+		struct module *owner = rdy->owner;
+
+		list_del_init(&rdy->list);
+		rdy->func(rdy);
+		module_put(owner);
+	}
+	spin_unlock_irqrestore(&lrng_ready_list_lock, flags);
+}
+
+void lrng_debug_report_seedlevel(const char *name)
+{
+#ifdef CONFIG_WARN_ALL_UNSEEDED_RANDOM
+	static void *previous = NULL;
+	void *caller = (void *) _RET_IP_;
+
+	if (READ_ONCE(previous) == caller)
+		return;
+
+	if (!lrng_state_min_seeded())
+		pr_notice("%pS %s called without reaching mimimally seeded level (available entropy %u)\n",
+			  caller, name, lrng_avail_entropy());
+
+	WRITE_ONCE(previous, caller);
+#endif
+}
+
+/************************ LRNG kernel input interfaces ************************/
+
+/**
+ * add_hwgenerator_randomness() - Interface for in-kernel drivers of true
+ * hardware RNGs.
+ *
+ * Those devices may produce endless random bits and will be throttled
+ * when our pool is full.
+ *
+ * @buffer: buffer holding the entropic data from HW noise sources to be used to
+ *	    insert into entropy pool.
+ * @count: length of buffer
+ * @entropy_bits: amount of entropy in buffer (value is in bits)
+ */
+void add_hwgenerator_randomness(const char *buffer, size_t count,
+				size_t entropy_bits)
+{
+	/*
+	 * Suspend writing if we are fully loaded with entropy.
+	 * We'll be woken up again once below lrng_write_wakeup_thresh,
+	 * or when the calling thread is about to terminate.
+	 */
+	wait_event_interruptible(lrng_write_wait,
+				lrng_need_entropy() ||
+				lrng_state_exseed_allow(lrng_noise_source_hw) ||
+				kthread_should_stop());
+	lrng_state_exseed_set(lrng_noise_source_hw, false);
+	lrng_pool_insert_aux(buffer, count, entropy_bits);
+}
+EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
+
+/**
+ * add_bootloader_randomness() - Handle random seed passed by bootloader.
+ *
+ * If the seed is trustworthy, it would be regarded as hardware RNGs. Otherwise
+ * it would be regarded as device data.
+ * The decision is controlled by CONFIG_RANDOM_TRUST_BOOTLOADER.
+ *
+ * @buf: buffer holding the entropic data from HW noise sources to be used to
+ *	 insert into entropy pool.
+ * @size: length of buffer
+ */
+void add_bootloader_randomness(const void *buf, unsigned int size)
+{
+	if (IS_ENABLED(CONFIG_RANDOM_TRUST_BOOTLOADER))
+		add_hwgenerator_randomness(buf, size, size * 8);
+	else
+		add_device_randomness(buf, size);
+}
+EXPORT_SYMBOL_GPL(add_bootloader_randomness);
+
+/*
+ * Callback for HID layer -- use the HID event values to stir the entropy pool
+ */
+void add_input_randomness(unsigned int type, unsigned int code,
+			  unsigned int value)
+{
+	static unsigned char last_value;
+
+	/* ignore autorepeat and the like */
+	if (value == last_value)
+		return;
+
+	last_value = value;
+
+	lrng_pcpu_array_add_u32((type << 4) ^ code ^ (code >> 4) ^ value);
+}
+EXPORT_SYMBOL_GPL(add_input_randomness);
+
+/**
+ * add_device_randomness() - Add device- or boot-specific data to the entropy
+ * pool to help initialize it.
+ *
+ * None of this adds any entropy; it is meant to avoid the problem of
+ * the entropy pool having similar initial state across largely
+ * identical devices.
+ *
+ * @buf: buffer holding the entropic data from HW noise sources to be used to
+ *	 insert into entropy pool.
+ * @size: length of buffer
+ */
+void add_device_randomness(const void *buf, unsigned int size)
+{
+	lrng_pool_insert_aux((u8 *)buf, size, 0);
+}
+EXPORT_SYMBOL(add_device_randomness);
+
+#ifdef CONFIG_BLOCK
+void rand_initialize_disk(struct gendisk *disk) { }
+void add_disk_randomness(struct gendisk *disk) { }
+EXPORT_SYMBOL(add_disk_randomness);
+#endif
+
+/**
+ * del_random_ready_callback() - Delete a previously registered readiness
+ * callback function.
+ *
+ * @rdy: callback definition that was registered initially
+ */
+void del_random_ready_callback(struct random_ready_callback *rdy)
+{
+	unsigned long flags;
+	struct module *owner = NULL;
+
+	spin_lock_irqsave(&lrng_ready_list_lock, flags);
+	if (!list_empty(&rdy->list)) {
+		list_del_init(&rdy->list);
+		owner = rdy->owner;
+	}
+	spin_unlock_irqrestore(&lrng_ready_list_lock, flags);
+
+	module_put(owner);
+}
+EXPORT_SYMBOL(del_random_ready_callback);
+
+/**
+ * add_random_ready_callback() - Add a callback function that will be invoked
+ * when the DRNG is mimimally seeded.
+ *
+ * @rdy: callback definition to be invoked when the LRNG is seeded
+ *
+ * Return:
+ * * 0 if callback is successfully added
+ * * -EALREADY if pool is already initialised (callback not called)
+ * * -ENOENT if module for callback is not alive
+ */
+int add_random_ready_callback(struct random_ready_callback *rdy)
+{
+	struct module *owner;
+	unsigned long flags;
+	int err = -EALREADY;
+
+	if (likely(lrng_state_min_seeded()))
+		return err;
+
+	owner = rdy->owner;
+	if (!try_module_get(owner))
+		return -ENOENT;
+
+	spin_lock_irqsave(&lrng_ready_list_lock, flags);
+	if (lrng_state_min_seeded())
+		goto out;
+
+	owner = NULL;
+
+	list_add(&rdy->list, &lrng_ready_list);
+	err = 0;
+
+out:
+	spin_unlock_irqrestore(&lrng_ready_list_lock, flags);
+
+	module_put(owner);
+
+	return err;
+}
+EXPORT_SYMBOL(add_random_ready_callback);
+
+/*********************** LRNG kernel output interfaces ************************/
+
+/**
+ * get_random_bytes() - Provider of cryptographic strong random numbers for
+ * kernel-internal usage.
+ *
+ * This function is appropriate for all in-kernel use cases. However,
+ * it will always use the ChaCha20 DRNG.
+ *
+ * @buf: buffer to store the random bytes
+ * @nbytes: size of the buffer
+ */
+void get_random_bytes(void *buf, int nbytes)
+{
+	lrng_drng_get_atomic((u8 *)buf, (u32)nbytes);
+	lrng_debug_report_seedlevel("get_random_bytes");
+}
+EXPORT_SYMBOL(get_random_bytes);
+
+/**
+ * get_random_bytes_full() - Provider of cryptographic strong random numbers
+ * for kernel-internal usage.
+ *
+ * This function is appropriate only for non-atomic use cases as this
+ * function may sleep. Though, it provides access to the full functionality
+ * of LRNG including the switchable DRNG support, that may support other
+ * DRNGs such as the SP800-90A DRBG.
+ *
+ * @buf: buffer to store the random bytes
+ * @nbytes: size of the buffer
+ */
+void get_random_bytes_full(void *buf, int nbytes)
+{
+	lrng_drng_get_sleep((u8 *)buf, (u32)nbytes);
+	lrng_debug_report_seedlevel("get_random_bytes_full");
+}
+EXPORT_SYMBOL(get_random_bytes_full);
+
+/**
+ * wait_for_random_bytes() - Wait for the LRNG to be seeded and thus
+ * guaranteed to supply cryptographically secure random numbers.
+ *
+ * This applies to: the /dev/urandom device, the get_random_bytes function,
+ * and the get_random_{u32,u64,int,long} family of functions. Using any of
+ * these functions without first calling this function forfeits the guarantee
+ * of security.
+ *
+ * Return:
+ * * 0 if the LRNG has been seeded.
+ * * -ERESTARTSYS if the function was interrupted by a signal.
+ */
+int wait_for_random_bytes(void)
+{
+	if (likely(lrng_state_min_seeded()))
+		return 0;
+	return wait_event_interruptible(lrng_init_wait,
+					lrng_state_min_seeded());
+}
+EXPORT_SYMBOL(wait_for_random_bytes);
+
+/**
+ * get_random_bytes_arch() - This function will use the architecture-specific
+ * hardware random number generator if it is available.
+ *
+ * The arch-specific hw RNG will almost certainly be faster than what we can
+ * do in software, but it is impossible to verify that it is implemented
+ * securely (as opposed, to, say, the AES encryption of a sequence number using
+ * a key known by the NSA).  So it's useful if we need the speed, but only if
+ * we're willing to trust the hardware manufacturer not to have put in a back
+ * door.
+ *
+ * @buf: buffer allocated by caller to store the random data in
+ * @nbytes: length of outbuf
+ *
+ * Return: number of bytes filled in.
+ */
+int __must_check get_random_bytes_arch(void *buf, int nbytes)
+{
+	u8 *p = buf;
+
+	while (nbytes) {
+		unsigned long v;
+		int chunk = min_t(int, nbytes, sizeof(unsigned long));
+
+		if (!arch_get_random_long(&v))
+			break;
+
+		memcpy(p, &v, chunk);
+		p += chunk;
+		nbytes -= chunk;
+	}
+
+	if (nbytes)
+		lrng_drng_get_atomic((u8 *)p, (u32)nbytes);
+
+	return nbytes;
+}
+EXPORT_SYMBOL(get_random_bytes_arch);
+
+/*
+ * Returns whether or not the LRNG has been seeded.
+ *
+ * Returns: true if the urandom pool has been seeded.
+ *          false if the urandom pool has not been seeded.
+ */
+bool rng_is_initialized(void)
+{
+	return lrng_state_operational();
+}
+EXPORT_SYMBOL(rng_is_initialized);
+
+/************************ LRNG user output interfaces *************************/
+
+static ssize_t lrng_read_common(char __user *buf, size_t nbytes)
+{
+	ssize_t ret = 0;
+	u8 tmpbuf[LRNG_DRNG_BLOCKSIZE] __aligned(LRNG_KCAPI_ALIGN);
+	u8 *tmp_large = NULL, *tmp = tmpbuf;
+	u32 tmplen = sizeof(tmpbuf);
+
+	if (nbytes == 0)
+		return 0;
+
+	/*
+	 * Satisfy large read requests -- as the common case are smaller
+	 * request sizes, such as 16 or 32 bytes, avoid a kmalloc overhead for
+	 * those by using the stack variable of tmpbuf.
+	 */
+	if (!CONFIG_BASE_SMALL && (nbytes > sizeof(tmpbuf))) {
+		tmplen = min_t(u32, nbytes, LRNG_DRNG_MAX_REQSIZE);
+		tmp_large = kmalloc(tmplen + LRNG_KCAPI_ALIGN, GFP_KERNEL);
+		if (!tmp_large)
+			tmplen = sizeof(tmpbuf);
+		else
+			tmp = PTR_ALIGN(tmp_large, LRNG_KCAPI_ALIGN);
+	}
+
+	while (nbytes) {
+		u32 todo = min_t(u32, nbytes, tmplen);
+		int rc = 0;
+
+		/* Reschedule if we received a large request. */
+		if ((tmp_large) && need_resched()) {
+			if (signal_pending(current)) {
+				if (ret == 0)
+					ret = -ERESTARTSYS;
+				break;
+			}
+			schedule();
+		}
+
+		rc = lrng_drng_get_sleep(tmp, todo);
+		if (rc <= 0) {
+			if (rc < 0)
+				ret = rc;
+			break;
+		}
+		if (copy_to_user(buf, tmp, rc)) {
+			ret = -EFAULT;
+			break;
+		}
+
+		nbytes -= rc;
+		buf += rc;
+		ret += rc;
+	}
+
+	/* Wipe data just returned from memory */
+	if (tmp_large)
+		kfree_sensitive(tmp_large);
+	else
+		memzero_explicit(tmpbuf, sizeof(tmpbuf));
+
+	return ret;
+}
+
+static ssize_t
+lrng_read_common_block(int nonblock, char __user *buf, size_t nbytes)
+{
+	if (nbytes == 0)
+		return 0;
+
+	if (unlikely(!lrng_state_operational())) {
+		int ret;
+
+		if (nonblock)
+			return -EAGAIN;
+
+		ret = wait_event_interruptible(lrng_init_wait,
+					       lrng_state_operational());
+		if (unlikely(ret))
+			return ret;
+	}
+
+	return lrng_read_common(buf, nbytes);
+}
+
+static ssize_t lrng_drng_read_block(struct file *file, char __user *buf,
+				     size_t nbytes, loff_t *ppos)
+{
+	return lrng_read_common_block(file->f_flags & O_NONBLOCK, buf, nbytes);
+}
+
+static __poll_t lrng_random_poll(struct file *file, poll_table *wait)
+{
+	__poll_t mask;
+
+	poll_wait(file, &lrng_init_wait, wait);
+	poll_wait(file, &lrng_write_wait, wait);
+	mask = 0;
+	if (lrng_state_operational())
+		mask |= EPOLLIN | EPOLLRDNORM;
+	if (lrng_need_entropy() ||
+	    lrng_state_exseed_allow(lrng_noise_source_user))
+		mask |= EPOLLOUT | EPOLLWRNORM;
+	return mask;
+}
+
+static ssize_t lrng_drng_write_common(const char __user *buffer, size_t count,
+				      u32 entropy_bits)
+{
+	ssize_t ret = 0;
+	u8 buf[64] __aligned(LRNG_KCAPI_ALIGN);
+	const char __user *p = buffer;
+	u32 orig_entropy_bits = entropy_bits;
+
+	if (!lrng_get_available())
+		return -EAGAIN;
+
+	count = min_t(size_t, count, INT_MAX);
+	while (count > 0) {
+		size_t bytes = min_t(size_t, count, sizeof(buf));
+		u32 ent = min_t(u32, bytes<<3, entropy_bits);
+
+		if (copy_from_user(&buf, p, bytes))
+			return -EFAULT;
+		/* Inject data into entropy pool */
+		lrng_pool_insert_aux(buf, bytes, ent);
+
+		count -= bytes;
+		p += bytes;
+		ret += bytes;
+		entropy_bits -= ent;
+
+		cond_resched();
+	}
+
+	/* Force reseed of DRNG during next data request. */
+	if (!orig_entropy_bits)
+		lrng_drng_force_reseed();
+
+	return ret;
+}
+
+static ssize_t lrng_drng_read(struct file *file, char __user *buf,
+			      size_t nbytes, loff_t *ppos)
+{
+	if (!lrng_state_min_seeded())
+		pr_notice_ratelimited("%s - use of insufficiently seeded DRNG (%zu bytes read)\n",
+				      current->comm, nbytes);
+	else if (!lrng_state_operational())
+		pr_debug_ratelimited("%s - use of not fully seeded DRNG (%zu bytes read)\n",
+				     current->comm, nbytes);
+
+	return lrng_read_common(buf, nbytes);
+}
+
+static ssize_t lrng_drng_write(struct file *file, const char __user *buffer,
+			       size_t count, loff_t *ppos)
+{
+	return lrng_drng_write_common(buffer, count, 0);
+}
+
+static long lrng_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
+{
+	u32 digestsize_bits;
+	int size, ent_count_bits;
+	int __user *p = (int __user *)arg;
+
+	switch (cmd) {
+	case RNDGETENTCNT:
+		ent_count_bits = lrng_avail_entropy();
+		if (put_user(ent_count_bits, p))
+			return -EFAULT;
+		return 0;
+	case RNDADDTOENTCNT:
+		if (!capable(CAP_SYS_ADMIN))
+			return -EPERM;
+		if (get_user(ent_count_bits, p))
+			return -EFAULT;
+		ent_count_bits = (int)lrng_avail_entropy() + ent_count_bits;
+		if (ent_count_bits < 0)
+			ent_count_bits = 0;
+		digestsize_bits = lrng_get_digestsize();
+		if (ent_count_bits > digestsize_bits)
+			ent_count_bits = digestsize_bits;
+		lrng_pool_set_entropy(ent_count_bits);
+		return 0;
+	case RNDADDENTROPY:
+		if (!capable(CAP_SYS_ADMIN))
+			return -EPERM;
+		if (get_user(ent_count_bits, p++))
+			return -EFAULT;
+		if (ent_count_bits < 0)
+			return -EINVAL;
+		if (get_user(size, p++))
+			return -EFAULT;
+		if (size < 0)
+			return -EINVAL;
+		lrng_state_exseed_set(lrng_noise_source_user, false);
+		/* there cannot be more entropy than data */
+		ent_count_bits = min(ent_count_bits, size<<3);
+		return lrng_drng_write_common((const char __user *)p, size,
+					      ent_count_bits);
+	case RNDZAPENTCNT:
+	case RNDCLEARPOOL:
+		/* Clear the entropy pool counter. */
+		if (!capable(CAP_SYS_ADMIN))
+			return -EPERM;
+		lrng_pool_set_entropy(0);
+		return 0;
+	case RNDRESEEDCRNG:
+		/*
+		 * We leave the capability check here since it is present
+		 * in the upstream's RNG implementation. Yet, user space
+		 * can trigger a reseed as easy as writing into /dev/random
+		 * or /dev/urandom where no privilege is needed.
+		 */
+		if (!capable(CAP_SYS_ADMIN))
+			return -EPERM;
+		/* Force a reseed of all DRNGs */
+		lrng_drng_force_reseed();
+		return 0;
+	default:
+		return -EINVAL;
+	}
+}
+
+static int lrng_fasync(int fd, struct file *filp, int on)
+{
+	return fasync_helper(fd, filp, on, &fasync);
+}
+
+const struct file_operations random_fops = {
+	.read  = lrng_drng_read_block,
+	.write = lrng_drng_write,
+	.poll  = lrng_random_poll,
+	.unlocked_ioctl = lrng_ioctl,
+	.compat_ioctl = compat_ptr_ioctl,
+	.fasync = lrng_fasync,
+	.llseek = noop_llseek,
+};
+
+const struct file_operations urandom_fops = {
+	.read  = lrng_drng_read,
+	.write = lrng_drng_write,
+	.unlocked_ioctl = lrng_ioctl,
+	.compat_ioctl = compat_ptr_ioctl,
+	.fasync = lrng_fasync,
+	.llseek = noop_llseek,
+};
+
+SYSCALL_DEFINE3(getrandom, char __user *, buf, size_t, count,
+		unsigned int, flags)
+{
+	if (flags & ~(GRND_NONBLOCK|GRND_RANDOM|GRND_INSECURE))
+		return -EINVAL;
+
+	/*
+	 * Requesting insecure and blocking randomness at the same time makes
+	 * no sense.
+	 */
+	if ((flags &
+	     (GRND_INSECURE|GRND_RANDOM)) == (GRND_INSECURE|GRND_RANDOM))
+		return -EINVAL;
+
+	if (count > INT_MAX)
+		count = INT_MAX;
+
+	if (flags & GRND_INSECURE)
+		return lrng_drng_read(NULL, buf, count, NULL);
+
+	return lrng_read_common_block(flags & GRND_NONBLOCK, buf, count);
+}
diff --git a/drivers/char/lrng/lrng_internal.h b/drivers/char/lrng/lrng_internal.h
new file mode 100644
index 000000000000..f9b80ce42341
--- /dev/null
+++ b/drivers/char/lrng/lrng_internal.h
@@ -0,0 +1,420 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * Copyright (C) 2018 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#ifndef _LRNG_INTERNAL_H
+#define _LRNG_INTERNAL_H
+
+#include <crypto/sha.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
+
+/*************************** General LRNG parameter ***************************/
+
+/* Security strength of LRNG -- this must match DRNG security strength */
+#define LRNG_DRNG_SECURITY_STRENGTH_BYTES 32
+#define LRNG_DRNG_SECURITY_STRENGTH_BITS (LRNG_DRNG_SECURITY_STRENGTH_BYTES * 8)
+#define LRNG_DRNG_BLOCKSIZE 64		/* Maximum of DRNG block sizes */
+
+/*
+ * SP800-90A defines a maximum request size of 1<<16 bytes. The given value is
+ * considered a safer margin.
+ *
+ * This value is allowed to be changed.
+ */
+#define LRNG_DRNG_MAX_REQSIZE		(1<<12)
+
+/*
+ * SP800-90A defines a maximum number of requests between reseeds of 2^48.
+ * The given value is considered a much safer margin, balancing requests for
+ * frequent reseeds with the need to conserve entropy. This value MUST NOT be
+ * larger than INT_MAX because it is used in an atomic_t.
+ *
+ * This value is allowed to be changed.
+ */
+#define LRNG_DRNG_RESEED_THRESH		(1<<20)
+
+/*
+ * Number of interrupts to be recorded to assume that DRNG security strength
+ * bits of entropy are received.
+ * Note: a value below the DRNG security strength should not be defined as this
+ *	 may imply the DRNG can never be fully seeded in case other noise
+ *	 sources are unavailable.
+ *
+ * This value is allowed to be changed.
+ */
+#define LRNG_IRQ_ENTROPY_BITS		LRNG_DRNG_SECURITY_STRENGTH_BITS
+
+/*
+ * Min required seed entropy is 128 bits covering the minimum entropy
+ * requirement of SP800-131A and the German BSI's TR02102.
+ *
+ * This value is allowed to be changed.
+ */
+#define LRNG_FULL_SEED_ENTROPY_BITS	LRNG_DRNG_SECURITY_STRENGTH_BITS
+#define LRNG_MIN_SEED_ENTROPY_BITS	128
+#define LRNG_INIT_ENTROPY_BITS		32
+
+/*
+ * Wakeup value
+ *
+ * This value is allowed to be changed but must not be larger than the
+ * digest size of the hash operation used update the aux_pool.
+ */
+#ifdef CONFIG_CRYPTO_LIB_SHA256
+# define LRNG_ATOMIC_DIGEST_SIZE	SHA256_DIGEST_SIZE
+#else
+# define LRNG_ATOMIC_DIGEST_SIZE	SHA1_DIGEST_SIZE
+#endif
+#define LRNG_WRITE_WAKEUP_ENTROPY	LRNG_ATOMIC_DIGEST_SIZE
+
+/*
+ * If the switching support is configured, we must provide support up to
+ * the largest digest size. Without switching support, we know it is only
+ * the built-in digest size.
+ */
+#ifdef CONFIG_LRNG_DRNG_SWITCH
+# define LRNG_MAX_DIGESTSIZE		64
+#else
+# define LRNG_MAX_DIGESTSIZE		LRNG_ATOMIC_DIGEST_SIZE
+#endif
+
+/*
+ * Oversampling factor of IRQ events to obtain
+ * LRNG_DRNG_SECURITY_STRENGTH_BYTES. This factor is used when a
+ * high-resolution time stamp is not available. In this case, jiffies and
+ * register contents are used to fill the entropy pool. These noise sources
+ * are much less entropic than the high-resolution timer. The entropy content
+ * is the entropy content assumed with LRNG_IRQ_ENTROPY_BITS divided by
+ * LRNG_IRQ_OVERSAMPLING_FACTOR.
+ *
+ * This value is allowed to be changed.
+ */
+#define LRNG_IRQ_OVERSAMPLING_FACTOR	10
+
+/*
+ * Alignmask which should cover all cipher implementations
+ * WARNING: If this is changed to a value larger than 8, manual
+ * alignment is necessary as older versions of GCC may not be capable
+ * of aligning stack variables at boundaries greater than 8.
+ * In this case, PTR_ALIGN must be used.
+ */
+#define LRNG_KCAPI_ALIGN		8
+
+/************************ Default DRNG implementation *************************/
+
+extern struct chacha20_state chacha20;
+extern const struct lrng_crypto_cb lrng_cc20_crypto_cb;
+void lrng_cc20_init_state(struct chacha20_state *state);
+void lrng_cc20_init_state_boot(struct chacha20_state *state);
+
+/********************************** /proc *************************************/
+
+static inline void lrng_pool_inc_numa_node(void) { }
+
+/****************************** LRNG interfaces *******************************/
+
+extern u32 lrng_write_wakeup_bits;
+extern int lrng_drng_reseed_max_time;
+
+void lrng_writer_wakeup(void);
+void lrng_init_wakeup(void);
+void lrng_debug_report_seedlevel(const char *name);
+void lrng_process_ready_list(void);
+
+/* External interface to use of the switchable DRBG inside the kernel */
+void get_random_bytes_full(void *buf, int nbytes);
+
+/************************** Jitter RNG Noise Source ***************************/
+
+#ifdef CONFIG_LRNG_JENT
+u32 lrng_get_jent(u8 *outbuf, unsigned int outbuflen);
+u32 lrng_jent_entropylevel(void);
+#else /* CONFIG_CRYPTO_JITTERENTROPY */
+static inline u32 lrng_get_jent(u8 *outbuf, unsigned int outbuflen) {return 0; }
+static inline u32 lrng_jent_entropylevel(void) { return 0; }
+#endif /* CONFIG_CRYPTO_JITTERENTROPY */
+
+/*************************** CPU-based Noise Source ***************************/
+
+u32 lrng_get_arch(u8 *outbuf);
+u32 lrng_slow_noise_req_entropy(u32 required_entropy_bits);
+
+/****************************** DRNG processing *******************************/
+
+/* Secondary DRNG state handle */
+struct lrng_drng {
+	void *drng;				/* DRNG handle */
+	void *hash;				/* Hash handle */
+	const struct lrng_crypto_cb *crypto_cb;	/* Crypto callbacks */
+	atomic_t requests;			/* Number of DRNG requests */
+	unsigned long last_seeded;		/* Last time it was seeded */
+	bool fully_seeded;			/* Is DRNG fully seeded? */
+	bool force_reseed;			/* Force a reseed */
+
+	/* Lock write operations on DRNG state, DRNG replacement of crypto_cb */
+	struct mutex lock;
+	spinlock_t spin_lock;
+	/* Lock hash replacement of crypto_cb */
+	rwlock_t hash_lock;
+};
+
+extern struct mutex lrng_crypto_cb_update;
+
+struct lrng_drng *lrng_drng_init_instance(void);
+struct lrng_drng *lrng_drng_atomic_instance(void);
+
+static __always_inline bool lrng_drng_is_atomic(struct lrng_drng *drng)
+{
+	return (drng->drng == lrng_drng_atomic_instance()->drng);
+}
+
+/* Lock the DRNG */
+static __always_inline void lrng_drng_lock(struct lrng_drng *drng,
+					   unsigned long *flags)
+	__acquires(&drng->spin_lock)
+{
+	/* Use spin lock in case the atomic DRNG context is used */
+	if (lrng_drng_is_atomic(drng)) {
+		spin_lock_irqsave(&drng->spin_lock, *flags);
+
+		/*
+		 * In case a lock transition happened while we were spinning,
+		 * catch this case and use the new lock type.
+		 */
+		if (!lrng_drng_is_atomic(drng)) {
+			spin_unlock_irqrestore(&drng->spin_lock, *flags);
+			__acquire(&drng->spin_lock);
+			mutex_lock(&drng->lock);
+		}
+	} else {
+		__acquire(&drng->spin_lock);
+		mutex_lock(&drng->lock);
+	}
+}
+
+/* Unlock the DRNG */
+static __always_inline void lrng_drng_unlock(struct lrng_drng *drng,
+					     unsigned long *flags)
+	__releases(&drng->spin_lock)
+{
+	if (lrng_drng_is_atomic(drng)) {
+		spin_unlock_irqrestore(&drng->spin_lock, *flags);
+	} else {
+		mutex_unlock(&drng->lock);
+		__release(&drng->spin_lock);
+	}
+}
+
+void lrng_reset(void);
+void lrng_drng_init_early(void);
+bool lrng_get_available(void);
+void lrng_set_available(void);
+void lrng_drng_reset(struct lrng_drng *drng);
+int lrng_drng_get_atomic(u8 *outbuf, u32 outbuflen);
+int lrng_drng_get_sleep(u8 *outbuf, u32 outbuflen);
+void lrng_drng_force_reseed(void);
+void lrng_drng_seed_work(struct work_struct *dummy);
+
+static inline struct lrng_drng **lrng_drng_instances(void) { return NULL; }
+static inline void lrng_drngs_numa_alloc(void) { return; }
+
+/************************** Entropy pool management ***************************/
+
+enum lrng_external_noise_source {
+	lrng_noise_source_hw,
+	lrng_noise_source_user
+};
+
+/* Status information about IRQ noise source */
+struct lrng_irq_info {
+	atomic_t num_events_thresh;	/* Reseed threshold */
+	atomic_t reseed_in_progress;	/* Flag for on executing reseed */
+	bool irq_highres_timer;	/* Is high-resolution timer available? */
+	u32 irq_entropy_bits;	/* LRNG_IRQ_ENTROPY_BITS? */
+};
+
+/*
+ * This is the entropy pool used by the slow noise source. Its size should
+ * be at least as large as LRNG_DRNG_SECURITY_STRENGTH_BITS.
+ *
+ * The aux pool array is aligned to 8 bytes to comfort the kernel crypto API
+ * cipher implementations of the hash functions used to read the pool: for some
+ * accelerated implementations, we need an alignment to avoid a realignment
+ * which involves memcpy(). The alignment to 8 bytes should satisfy all crypto
+ * implementations.
+ */
+struct lrng_pool {
+	/*
+	 * Storage for aux data - hash output buffer
+	 */
+	u8 aux_pool[LRNG_MAX_DIGESTSIZE];
+	atomic_t aux_entropy_bits;
+	/* All NUMA DRNGs seeded? */
+	bool all_online_numa_node_seeded;
+
+	/* Digest size of used hash */
+	atomic_t digestsize;
+	/* IRQ noise source status info */
+	struct lrng_irq_info irq_info;
+
+	/* Serialize read of entropy pool and update of aux pool */
+	spinlock_t lock;
+};
+
+u32 lrng_entropy_to_data(u32 entropy_bits);
+u32 lrng_data_to_entropy(u32 irqnum);
+u32 lrng_avail_aux_entropy(void);
+void lrng_set_digestsize(u32 digestsize);
+u32 lrng_get_digestsize(void);
+
+/* Obtain the security strength of the LRNG in bits */
+static inline u32 lrng_security_strength(void)
+{
+	/*
+	 * We use a hash to read the entropy in the entropy pool. According to
+	 * SP800-90B table 1, the entropy can be at most the digest size.
+	 * Considering this together with the last sentence in section 3.1.5.1.2
+	 * the security strength of a (approved) hash is equal to its output
+	 * size. On the other hand the entropy cannot be larger than the
+	 * security strength of the used DRBG.
+	 */
+	return min_t(u32, LRNG_FULL_SEED_ENTROPY_BITS,
+		     lrng_get_digestsize());
+}
+
+void lrng_set_entropy_thresh(u32 new);
+void lrng_reset_state(void);
+
+void lrng_pcpu_reset(void);
+u32 lrng_pcpu_avail_irqs(void);
+
+static inline u32 lrng_pcpu_avail_entropy(void)
+{
+	return lrng_data_to_entropy(lrng_pcpu_avail_irqs());
+}
+
+static inline u32 lrng_avail_entropy(void)
+{
+	return lrng_pcpu_avail_entropy() + lrng_avail_aux_entropy();
+}
+
+u32 lrng_pcpu_pool_hash(struct lrng_drng *drng, struct lrng_pool *pool,
+			u8 *outbuf, u32 requested_bits, bool fully_seeded);
+void lrng_pcpu_array_add_u32(u32 data);
+
+bool lrng_state_exseed_allow(enum lrng_external_noise_source source);
+void lrng_state_exseed_set(enum lrng_external_noise_source source, bool type);
+void lrng_state_init_seed_work(void);
+bool lrng_state_min_seeded(void);
+bool lrng_state_fully_seeded(void);
+bool lrng_state_operational(void);
+
+int lrng_pool_trylock(void);
+void lrng_pool_unlock(void);
+void lrng_pool_all_numa_nodes_seeded(void);
+bool lrng_pool_highres_timer(void);
+void lrng_pool_set_entropy(u32 entropy_bits);
+int lrng_pool_insert_aux(const u8 *inbuf, u32 inbuflen, u32 entropy_bits);
+void lrng_pool_add_irq(void);
+
+struct entropy_buf {
+	u8 a[LRNG_DRNG_SECURITY_STRENGTH_BYTES];
+	u8 b[LRNG_DRNG_SECURITY_STRENGTH_BYTES];
+	u8 c[LRNG_DRNG_SECURITY_STRENGTH_BYTES];
+	u32 now;
+};
+
+int lrng_fill_seed_buffer(struct lrng_drng *drng,
+			  struct entropy_buf *entropy_buf);
+void lrng_init_ops(u32 seed_bits);
+
+/************************** Health Test linking code **************************/
+
+enum lrng_health_res {
+	lrng_health_pass,		/* Health test passes on time stamp */
+	lrng_health_fail_use,		/* Time stamp unhealthy, but mix in */
+	lrng_health_fail_drop		/* Time stamp unhealthy, drop it */
+};
+
+#ifdef CONFIG_LRNG_HEALTH_TESTS
+bool lrng_sp80090b_startup_complete(void);
+bool lrng_sp80090b_compliant(void);
+
+enum lrng_health_res lrng_health_test(u32 now_time);
+void lrng_health_disable(void);
+
+#else	/* CONFIG_LRNG_HEALTH_TESTS */
+static inline bool lrng_sp80090b_startup_complete(void) { return true; }
+static inline bool lrng_sp80090b_compliant(void) { return false; }
+
+static inline enum lrng_health_res
+lrng_health_test(u32 now_time) { return lrng_health_pass; }
+static inline void lrng_health_disable(void) { }
+#endif	/* CONFIG_LRNG_HEALTH_TESTS */
+
+/****************************** Helper code ***********************************/
+
+static inline u32 atomic_read_u32(atomic_t *v)
+{
+	return (u32)atomic_read(v);
+}
+
+/*************************** Auxiliary functions ******************************/
+
+void invalidate_batched_entropy(void);
+
+/***************************** Testing code ***********************************/
+
+#ifdef CONFIG_LRNG_RAW_HIRES_ENTROPY
+bool lrng_raw_hires_entropy_store(u32 value);
+#else	/* CONFIG_LRNG_RAW_HIRES_ENTROPY */
+static inline bool lrng_raw_hires_entropy_store(u32 value) { return false; }
+#endif	/* CONFIG_LRNG_RAW_HIRES_ENTROPY */
+
+#ifdef CONFIG_LRNG_RAW_JIFFIES_ENTROPY
+bool lrng_raw_jiffies_entropy_store(u32 value);
+#else	/* CONFIG_LRNG_RAW_JIFFIES_ENTROPY */
+static inline bool lrng_raw_jiffies_entropy_store(u32 value) { return false; }
+#endif	/* CONFIG_LRNG_RAW_JIFFIES_ENTROPY */
+
+#ifdef CONFIG_LRNG_RAW_IRQ_ENTROPY
+bool lrng_raw_irq_entropy_store(u32 value);
+#else	/* CONFIG_LRNG_RAW_IRQ_ENTROPY */
+static inline bool lrng_raw_irq_entropy_store(u32 value) { return false; }
+#endif	/* CONFIG_LRNG_RAW_IRQ_ENTROPY */
+
+#ifdef CONFIG_LRNG_RAW_IRQFLAGS_ENTROPY
+bool lrng_raw_irqflags_entropy_store(u32 value);
+#else	/* CONFIG_LRNG_RAW_IRQFLAGS_ENTROPY */
+static inline bool lrng_raw_irqflags_entropy_store(u32 value) { return false; }
+#endif	/* CONFIG_LRNG_RAW_IRQFLAGS_ENTROPY */
+
+#ifdef CONFIG_LRNG_RAW_RETIP_ENTROPY
+bool lrng_raw_retip_entropy_store(u32 value);
+#else	/* CONFIG_LRNG_RAW_RETIP_ENTROPY */
+static inline bool lrng_raw_retip_entropy_store(u32 value) { return false; }
+#endif	/* CONFIG_LRNG_RAW_RETIP_ENTROPY */
+
+#ifdef CONFIG_LRNG_RAW_REGS_ENTROPY
+bool lrng_raw_regs_entropy_store(u32 value);
+#else	/* CONFIG_LRNG_RAW_REGS_ENTROPY */
+static inline bool lrng_raw_regs_entropy_store(u32 value) { return false; }
+#endif	/* CONFIG_LRNG_RAW_REGS_ENTROPY */
+
+#ifdef CONFIG_LRNG_RAW_ARRAY
+bool lrng_raw_array_entropy_store(u32 value);
+#else	/* CONFIG_LRNG_RAW_ARRAY */
+static inline bool lrng_raw_array_entropy_store(u32 value) { return false; }
+#endif	/* CONFIG_LRNG_RAW_ARRAY */
+
+#ifdef CONFIG_LRNG_IRQ_PERF
+bool lrng_perf_time(u32 start);
+#else /* CONFIG_LRNG_IRQ_PERF */
+static inline bool lrng_perf_time(u32 start) { return false; }
+#endif /*CONFIG_LRNG_IRQ_PERF */
+
+#endif /* _LRNG_INTERNAL_H */
diff --git a/drivers/char/lrng/lrng_pool.c b/drivers/char/lrng/lrng_pool.c
new file mode 100644
index 000000000000..4a7cc94884e4
--- /dev/null
+++ b/drivers/char/lrng/lrng_pool.c
@@ -0,0 +1,457 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG Entropy pool management
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <asm/irq_regs.h>
+#include <linux/lrng.h>
+#include <linux/percpu.h>
+#include <linux/random.h>
+#include <linux/utsname.h>
+#include <linux/workqueue.h>
+
+#include "lrng_internal.h"
+
+struct lrng_state {
+	bool lrng_operational;		/* Is DRNG operational? */
+	bool lrng_fully_seeded;		/* Is DRNG fully seeded? */
+	bool lrng_min_seeded;		/* Is DRNG minimally seeded? */
+
+	/*
+	 * To ensure that external entropy providers cannot dominate the
+	 * internal noise sources but yet cannot be dominated by internal
+	 * noise sources, the following booleans are intended to allow
+	 * external to provide seed once when a DRNG reseed occurs. This
+	 * triggering of external noise source is performed even when the
+	 * entropy pool has sufficient entropy.
+	 */
+	bool lrng_seed_hw;		/* Allow HW to provide seed */
+	bool lrng_seed_user;		/* Allow user space to provide seed */
+
+	struct work_struct lrng_seed_work;	/* (re)seed work queue */
+};
+
+static struct lrng_pool lrng_pool __aligned(LRNG_KCAPI_ALIGN) = {
+	.aux_entropy_bits	= ATOMIC_INIT(0),
+	.digestsize		= ATOMIC_INIT(LRNG_ATOMIC_DIGEST_SIZE),
+	.irq_info		= {
+		.irq_entropy_bits	= LRNG_IRQ_ENTROPY_BITS,
+		.num_events_thresh	= ATOMIC_INIT(LRNG_INIT_ENTROPY_BITS),
+		/* Sample IRQ pointer data at least during boot */
+		.irq_highres_timer	= false },
+	.lock			= __SPIN_LOCK_UNLOCKED(lrng_pool.lock)
+};
+
+static struct lrng_state lrng_state = { false, false, false, true, true };
+
+/********************************** Helper ***********************************/
+
+/* External entropy provider is allowed to provide seed data */
+bool lrng_state_exseed_allow(enum lrng_external_noise_source source)
+{
+	if (source == lrng_noise_source_hw)
+		return lrng_state.lrng_seed_hw;
+	return lrng_state.lrng_seed_user;
+}
+
+/* Enable / disable external entropy provider to furnish seed */
+void lrng_state_exseed_set(enum lrng_external_noise_source source, bool type)
+{
+	if (source == lrng_noise_source_hw)
+		lrng_state.lrng_seed_hw = type;
+	else
+		lrng_state.lrng_seed_user = type;
+}
+
+static inline void lrng_state_exseed_allow_all(void)
+{
+	lrng_state_exseed_set(lrng_noise_source_hw, true);
+	lrng_state_exseed_set(lrng_noise_source_user, true);
+}
+
+/* Initialize the seed work queue */
+void lrng_state_init_seed_work(void)
+{
+	INIT_WORK(&lrng_state.lrng_seed_work, lrng_drng_seed_work);
+}
+
+/* Convert entropy in bits into number of IRQs with the same entropy content. */
+u32 lrng_entropy_to_data(u32 entropy_bits)
+{
+	return ((entropy_bits * lrng_pool.irq_info.irq_entropy_bits) /
+		LRNG_DRNG_SECURITY_STRENGTH_BITS);
+}
+
+/* Convert number of IRQs into entropy value. */
+u32 lrng_data_to_entropy(u32 irqnum)
+{
+	return ((irqnum * LRNG_DRNG_SECURITY_STRENGTH_BITS) /
+		lrng_pool.irq_info.irq_entropy_bits);
+}
+
+/* Entropy in bits present in aux pool */
+u32 lrng_avail_aux_entropy(void)
+{
+	/* Cap available entropy with max entropy */
+	return min_t(u32, atomic_read_u32(&lrng_pool.digestsize) << 3,
+		     atomic_read_u32(&lrng_pool.aux_entropy_bits));
+}
+
+/* Set the digest size of the used hash in bytes */
+void lrng_set_digestsize(u32 digestsize)
+{
+	atomic_set(&lrng_pool.digestsize, digestsize);
+}
+
+/* Obtain the digest size provided by the used hash in bits */
+u32 lrng_get_digestsize(void)
+{
+	return atomic_read_u32(&lrng_pool.digestsize) << 3;
+}
+
+/* Set new entropy threshold for reseeding during boot */
+void lrng_set_entropy_thresh(u32 new_entropy_bits)
+{
+	atomic_set(&lrng_pool.irq_info.num_events_thresh,
+		   lrng_entropy_to_data(new_entropy_bits));
+}
+
+/*
+ * Reading of the LRNG pool is only allowed by one caller. The reading is
+ * only performed to (re)seed DRNGs. Thus, if this "lock" is already taken,
+ * the reseeding operation is in progress. The caller is not intended to wait
+ * but continue with its other operation.
+ */
+int lrng_pool_trylock(void)
+{
+	return atomic_cmpxchg(&lrng_pool.irq_info.reseed_in_progress, 0, 1);
+}
+
+void lrng_pool_unlock(void)
+{
+	atomic_set(&lrng_pool.irq_info.reseed_in_progress, 0);
+}
+
+/*
+ * Reset LRNG state - the entropy counters are reset, but the data that may
+ * or may not have entropy remains in the pools as this data will not hurt.
+ */
+void lrng_reset_state(void)
+{
+	atomic_set(&lrng_pool.aux_entropy_bits, 0);
+	lrng_pcpu_reset();
+	lrng_state.lrng_operational = false;
+	lrng_state.lrng_fully_seeded = false;
+	lrng_state.lrng_min_seeded = false;
+	lrng_pool.all_online_numa_node_seeded = false;
+	pr_debug("reset LRNG\n");
+}
+
+/* Set flag that all DRNGs are fully seeded */
+void lrng_pool_all_numa_nodes_seeded(void)
+{
+	lrng_pool.all_online_numa_node_seeded = true;
+}
+
+/* Return boolean whether LRNG reached minimally seed level */
+bool lrng_state_min_seeded(void)
+{
+	return lrng_state.lrng_min_seeded;
+}
+
+/* Return boolean whether LRNG reached fully seed level */
+bool lrng_state_fully_seeded(void)
+{
+	return lrng_state.lrng_fully_seeded;
+}
+
+/* Return boolean whether LRNG is considered fully operational */
+bool lrng_state_operational(void)
+{
+	return lrng_state.lrng_operational;
+}
+
+/* Return boolean whether LRNG identified presence of high-resolution timer */
+bool lrng_pool_highres_timer(void)
+{
+	return lrng_pool.irq_info.irq_highres_timer;
+}
+
+/* Set entropy content in user-space controllable aux pool */
+void lrng_pool_set_entropy(u32 entropy_bits)
+{
+	atomic_set(&lrng_pool.aux_entropy_bits, entropy_bits);
+}
+
+static void lrng_pool_configure(bool highres_timer, u32 irq_entropy_bits)
+{
+	struct lrng_irq_info *irq_info = &lrng_pool.irq_info;
+
+	irq_info->irq_highres_timer = highres_timer;
+	if (irq_info->irq_entropy_bits != irq_entropy_bits) {
+		irq_info->irq_entropy_bits = irq_entropy_bits;
+		/* Reset the threshold based on new oversampling factor. */
+		lrng_set_entropy_thresh(atomic_read_u32(
+						&irq_info->num_events_thresh));
+	}
+}
+
+static int __init lrng_init_time_source(void)
+{
+	if (random_get_entropy() || random_get_entropy()) {
+		/*
+		 * As the highres timer is identified here, previous interrupts
+		 * obtained during boot time are treated like a lowres-timer
+		 * would have been present.
+		 */
+		lrng_pool_configure(true, LRNG_IRQ_ENTROPY_BITS);
+	} else {
+		lrng_health_disable();
+		lrng_pool_configure(false, LRNG_IRQ_ENTROPY_BITS *
+					   LRNG_IRQ_OVERSAMPLING_FACTOR);
+		pr_warn("operating without high-resolution timer and applying IRQ oversampling factor %u\n",
+			LRNG_IRQ_OVERSAMPLING_FACTOR);
+	}
+
+	return 0;
+}
+
+core_initcall(lrng_init_time_source);
+
+/**
+ * lrng_init_ops() - Set seed stages of LRNG
+ *
+ * Set the slow noise source reseed trigger threshold. The initial threshold
+ * is set to the minimum data size that can be read from the pool: a word. Upon
+ * reaching this value, the next seed threshold of 128 bits is set followed
+ * by 256 bits.
+ *
+ * @entropy_bits: size of entropy currently injected into DRNG
+ */
+void lrng_init_ops(u32 seed_bits)
+{
+	struct lrng_state *state = &lrng_state;
+
+	if (state->lrng_operational)
+		return;
+
+	/* DRNG is seeded with full security strength */
+	if (state->lrng_fully_seeded) {
+		state->lrng_operational = lrng_sp80090b_startup_complete();
+		lrng_process_ready_list();
+		lrng_init_wakeup();
+	} else if (seed_bits >= lrng_security_strength()) {
+		invalidate_batched_entropy();
+		state->lrng_fully_seeded = true;
+		state->lrng_operational = lrng_sp80090b_startup_complete();
+		state->lrng_min_seeded = true;
+		pr_info("LRNG fully seeded with %u bits of entropy\n",
+			seed_bits);
+		lrng_set_entropy_thresh(lrng_security_strength());
+		lrng_process_ready_list();
+		lrng_init_wakeup();
+
+	} else if (!state->lrng_min_seeded) {
+
+		/* DRNG is seeded with at least 128 bits of entropy */
+		if (seed_bits >= LRNG_MIN_SEED_ENTROPY_BITS) {
+			invalidate_batched_entropy();
+			state->lrng_min_seeded = true;
+			pr_info("LRNG minimally seeded with %u bits of entropy\n",
+				seed_bits);
+			lrng_set_entropy_thresh(
+				lrng_slow_noise_req_entropy(
+						lrng_security_strength()));
+			lrng_process_ready_list();
+			lrng_init_wakeup();
+
+		/* DRNG is seeded with at least LRNG_INIT_ENTROPY_BITS bits */
+		} else if (seed_bits >= LRNG_INIT_ENTROPY_BITS) {
+			pr_info("LRNG initial entropy level %u bits of entropy\n",
+				seed_bits);
+			lrng_set_entropy_thresh(
+				lrng_slow_noise_req_entropy(
+					LRNG_MIN_SEED_ENTROPY_BITS));
+		}
+	}
+}
+
+int __init rand_initialize(void)
+{
+	struct seed {
+		ktime_t time;
+		unsigned long data[(LRNG_MAX_DIGESTSIZE /
+				    sizeof(unsigned long))];
+		struct new_utsname utsname;
+	} seed __aligned(LRNG_KCAPI_ALIGN);
+	unsigned int i;
+
+	lrng_drng_init_early();
+
+	BUILD_BUG_ON(LRNG_MAX_DIGESTSIZE % sizeof(unsigned long));
+
+	seed.time = ktime_get_real();
+
+	for (i = 0; i < ARRAY_SIZE(seed.data); i++) {
+		if (!arch_get_random_seed_long_early(&(seed.data[i])) &&
+		    !arch_get_random_long_early(&seed.data[i]))
+			seed.data[i] = random_get_entropy();
+	}
+	memcpy(&seed.utsname, utsname(), sizeof(*(utsname())));
+
+	lrng_pool_insert_aux((u8 *)&seed, sizeof(seed), 0);
+	memzero_explicit(&seed, sizeof(seed));
+
+	return 0;
+}
+
+/*
+ * Insert data into auxiliary pool by hashing the input data together with
+ * the auxiliary pool. The message digest is the new state of the auxiliary
+ * pool.
+ */
+int lrng_pool_insert_aux(const u8 *inbuf, u32 inbuflen, u32 entropy_bits)
+{
+	SHASH_DESC_ON_STACK(shash, NULL);
+	struct lrng_drng *drng = lrng_drng_init_instance();
+	const struct lrng_crypto_cb *crypto_cb;
+	struct lrng_pool *pool = &lrng_pool;
+	unsigned long flags, flags2;
+	void *hash;
+	u32 digestsize;
+	int ret;
+
+	if (entropy_bits > (inbuflen << 3))
+		entropy_bits = (inbuflen << 3);
+
+	read_lock_irqsave(&drng->hash_lock, flags);
+
+	crypto_cb = drng->crypto_cb;
+	hash = drng->hash;
+	digestsize = crypto_cb->lrng_hash_digestsize(hash);
+
+	spin_lock_irqsave(&pool->lock, flags2);
+	ret = crypto_cb->lrng_hash_init(shash, hash) ?:
+	      /* Hash auxiliary pool ... */
+	      crypto_cb->lrng_hash_update(shash, pool->aux_pool, digestsize) ?:
+	      /* ... together with input data ... */
+	      crypto_cb->lrng_hash_update(shash, inbuf, inbuflen) ?:
+	      /* ... to form mew auxiliary pool state. */
+	      crypto_cb->lrng_hash_final(shash, pool->aux_pool);
+	if (ret)
+		goto out;
+
+	/*
+	 * Cap the available entropy to the hash output size compliant to
+	 * SP800-90B section 3.1.5.1 table 1.
+	 */
+	entropy_bits += atomic_read_u32(&pool->aux_entropy_bits);
+	if (entropy_bits > digestsize << 3)
+		entropy_bits = digestsize << 3;
+	atomic_set(&pool->aux_entropy_bits, entropy_bits);
+
+out:
+	spin_unlock_irqrestore(&pool->lock, flags2);
+	read_unlock_irqrestore(&drng->hash_lock, flags);
+
+	return ret;
+}
+
+/* Hot code path during boot - mix data into entropy pool during boot */
+void lrng_pool_add_irq(void)
+{
+	/*
+	 * Once all DRNGs are fully seeded, the interrupt noise
+	 * sources will not trigger any reseeding any more.
+	 */
+	if (likely(lrng_pool.all_online_numa_node_seeded))
+		return;
+
+	/* Only try to reseed if the DRNG is alive. */
+	if (!lrng_get_available())
+		return;
+
+	/* Only trigger the DRNG reseed if we have collected enough IRQs. */
+	if (lrng_pcpu_avail_irqs() <
+	    atomic_read_u32(&lrng_pool.irq_info.num_events_thresh))
+		return;
+
+	/* Ensure that the seeding only occurs once at any given time. */
+	if (lrng_pool_trylock())
+		return;
+
+	/* Seed the DRNG with IRQ noise. */
+	schedule_work(&lrng_state.lrng_seed_work);
+}
+
+/************************* Get data from entropy pool *************************/
+
+static u32 lrng_get_pool(struct lrng_drng *drng, u8 *outbuf,
+			 u32 requested_entropy_bits)
+{
+	struct lrng_pool *pool = &lrng_pool;
+	struct lrng_state *state = &lrng_state;
+	unsigned long flags;
+
+	/* We operate on the non-atomic part of the pool */
+	spin_lock_irqsave(&pool->lock, flags);
+	requested_entropy_bits = lrng_pcpu_pool_hash(drng, &lrng_pool,
+						     outbuf,
+						     requested_entropy_bits,
+						     state->lrng_fully_seeded);
+	spin_unlock_irqrestore(&pool->lock, flags);
+
+	return requested_entropy_bits;
+}
+
+/* Fill the seed buffer with data from the noise sources */
+int lrng_fill_seed_buffer(struct lrng_drng *drng,
+			  struct entropy_buf *entropy_buf)
+{
+	struct lrng_state *state = &lrng_state;
+	u32 total_entropy_bits = 0;
+
+	/* Guarantee that requested bits is a multiple of bytes */
+	BUILD_BUG_ON(LRNG_DRNG_SECURITY_STRENGTH_BITS % 8);
+
+	/* Require at least 128 bits of entropy for any reseed. */
+	if (state->lrng_fully_seeded &&
+	    (lrng_avail_entropy() <
+	     lrng_slow_noise_req_entropy(LRNG_MIN_SEED_ENTROPY_BITS)))
+		goto wakeup;
+
+	/*
+	 * Concatenate the output of the noise sources. This would be the
+	 * spot to add an entropy extractor logic if desired. Note, this
+	 * has the ability to collect entropy equal or larger than the DRNG
+	 * strength.
+	 */
+	total_entropy_bits = lrng_get_pool(drng, entropy_buf->a,
+					   LRNG_DRNG_SECURITY_STRENGTH_BITS);
+	total_entropy_bits += lrng_get_arch(entropy_buf->b);
+	total_entropy_bits += lrng_get_jent(entropy_buf->c,
+					    LRNG_DRNG_SECURITY_STRENGTH_BYTES);
+
+	/* also reseed the DRNG with the current time stamp */
+	entropy_buf->now = random_get_entropy();
+
+	/* allow external entropy provider to provide seed */
+	lrng_state_exseed_allow_all();
+
+wakeup:
+	/*
+	 * Shall we wake up user space writers? This location covers
+	 * ensures that the user space provider does not dominate the internal
+	 * noise sources since in case the first call of this function finds
+	 * sufficient entropy in the entropy pool, it will not trigger the
+	 * wakeup. This implies that when the next /dev/urandom read happens,
+	 * the entropy pool is drained.
+	 */
+	lrng_writer_wakeup();
+
+	return total_entropy_bits;
+}
diff --git a/drivers/char/lrng/lrng_sw_noise.c b/drivers/char/lrng/lrng_sw_noise.c
new file mode 100644
index 000000000000..2ea613499dfd
--- /dev/null
+++ b/drivers/char/lrng/lrng_sw_noise.c
@@ -0,0 +1,466 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG Slow Noise Source: Interrupt data collection and random data generation
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <asm/irq_regs.h>
+#include <asm/ptrace.h>
+#include <crypto/hash.h>
+#include <linux/lrng.h>
+#include <linux/random.h>
+
+#include "lrng_internal.h"
+#include "lrng_sw_noise.h"
+
+/* Per-CPU array holding concatenated entropy events */
+static DEFINE_PER_CPU(u32 [LRNG_DATA_ARRAY_SIZE], lrng_pcpu_array)
+						__aligned(LRNG_KCAPI_ALIGN);
+static DEFINE_PER_CPU(u32, lrng_pcpu_array_ptr) = 0;
+static DEFINE_PER_CPU(atomic_t, lrng_pcpu_array_irqs) = ATOMIC_INIT(0);
+
+/* Per-CPU entropy pool with compressed entropy events */
+static DEFINE_PER_CPU(u8 [LRNG_MAX_DIGESTSIZE], lrng_pcpu_pool)
+						__aligned(LRNG_KCAPI_ALIGN);
+/*
+ * Lock to allow other CPUs to read the pool - as this is only done during
+ * reseed which is infrequent, this lock is hardly contended.
+ */
+static DEFINE_PER_CPU(spinlock_t, lrng_pcpu_lock);
+static DEFINE_PER_CPU(bool, lrng_pcpu_lock_init) = false;
+
+static inline bool lrng_pcpu_pool_online(int cpu)
+{
+	return per_cpu(lrng_pcpu_lock_init, cpu);
+}
+
+/*
+ * Reset all per-CPU pools - reset entropy estimator but leave the pool data
+ * that may or may not have entropy unchanged.
+ */
+void lrng_pcpu_reset(void)
+{
+	int cpu;
+
+	for_each_online_cpu(cpu)
+		atomic_set(per_cpu_ptr(&lrng_pcpu_array_irqs, cpu), 0);
+}
+
+/* Return number of unused IRQs present in all per-CPU pools. */
+u32 lrng_pcpu_avail_irqs(void)
+{
+	u32 digestsize_irqs, irq = 0;
+	int cpu;
+
+	/* Obtain the cap of maximum numbers of IRQs we count */
+	digestsize_irqs = lrng_entropy_to_data(lrng_get_digestsize());
+
+	for_each_online_cpu(cpu) {
+		if (!lrng_pcpu_pool_online(cpu))
+			continue;
+		irq += min_t(u32, digestsize_irqs,
+			     atomic_read_u32(per_cpu_ptr(&lrng_pcpu_array_irqs,
+							 cpu)));
+	}
+
+	return irq;
+}
+
+/**
+ * Hash all per-CPU pools and the auxiliary pool to form a new auxiliary pool
+ * state. The message digest is at the same time the new state of the aux pool
+ * to ensure backtracking resistance and the seed data used for seeding a DRNG.
+ * The function will only copy as much data as entropy is available into the
+ * caller-provided output buffer.
+ *
+ * This function handles the translation from the number of received interrupts
+ * into an entropy statement. The conversion depends on LRNG_IRQ_ENTROPY_BITS
+ * which defines how many interrupts must be received to obtain 256 bits of
+ * entropy. With this value, the function lrng_data_to_entropy converts a given
+ * data size (received interrupts, requested amount of data, etc.) into an
+ * entropy statement. lrng_entropy_to_data does the reverse.
+ *
+ * Both functions are agnostic about the type of data: when the number of
+ * interrupts is processed by these functions, the resulting entropy value is in
+ * bits as we assume the entropy of interrupts is measured in bits. When data is
+ * processed, the entropy value is in bytes as the data is measured in bytes.
+ *
+ * lrng_pool->lock must be held by caller as we update a non-atomic pool part.
+ *
+ * @drng: DRNG state providing the crypto callbacks to use
+ * @pool: global entropy pool holding the aux pool
+ * @outbuf: buffer to store data in with size LRNG_DRNG_SECURITY_STRENGTH_BYTES
+ * @requested_bits: amount of data to be generated
+ * @fully_seeded: indicator whether LRNG is fully seeded
+ * @return: amount of collected entropy in bits.
+ */
+u32 lrng_pcpu_pool_hash(struct lrng_drng *drng, struct lrng_pool *pool,
+			u8 *outbuf, u32 requested_bits, bool fully_seeded)
+{
+	SHASH_DESC_ON_STACK(shash, NULL);
+	const struct lrng_crypto_cb *crypto_cb;
+	unsigned long flags;
+	u32 digestsize_bits, found_ent_bits, found_irqs, unused_bits = 0,
+	    collected_ent_bits = 0, collected_irqs = 0, requested_irqs,
+	    digestsize_irqs;
+	int ret, cpu;
+	void *hash;
+
+	read_lock_irqsave(&drng->hash_lock, flags);
+
+	crypto_cb = drng->crypto_cb;
+	hash = drng->hash;
+	digestsize_bits = crypto_cb->lrng_hash_digestsize(hash) << 3;
+	digestsize_irqs = lrng_entropy_to_data(digestsize_bits);
+
+	ret = crypto_cb->lrng_hash_init(shash, hash);
+	if (ret)
+		goto err;
+
+	/* Deduct entropy counter from aux pool */
+	found_ent_bits = atomic_xchg_relaxed(&pool->aux_entropy_bits, 0);
+	/* Cap entropy by security strength of used digest */
+	found_ent_bits = min_t(u32, digestsize_bits, found_ent_bits);
+
+	/* Harvest entropy from aux pool */
+	ret = crypto_cb->lrng_hash_update(shash, (u8 *)pool, sizeof(*pool));
+	if (ret)
+		goto err;
+
+	/* We collected that amount of entropy */
+	collected_ent_bits += found_ent_bits;
+	/* We collected too much entropy and put the overflow back */
+	if (collected_ent_bits > requested_bits) {
+		/* Amount of bits we collected too much */
+		unused_bits = collected_ent_bits - requested_bits;
+
+		/* Store that for logging */
+		found_ent_bits -= unused_bits;
+		/* Put entropy back */
+		atomic_add(found_ent_bits, &pool->aux_entropy_bits);
+		/* Fix collected entropy */
+		collected_ent_bits = requested_bits;
+	}
+	pr_debug("%u bits of entropy used from aux pool, %u bits of entropy remaining\n",
+		 found_ent_bits, unused_bits);
+
+	requested_irqs = lrng_entropy_to_data(requested_bits -
+					      collected_ent_bits);
+
+	/*
+	 * Harvest entropy from each per-CPU hash state - even though we may
+	 * have collected sufficient entropy, we will hash all per-CPU pools.
+	 */
+	for_each_online_cpu(cpu) {
+		unsigned long flags2;
+		spinlock_t *lock = per_cpu_ptr(&lrng_pcpu_lock, cpu);
+		u32 pcpu_unused_irqs = 0;
+		u8 *pcpu_pool = per_cpu_ptr(lrng_pcpu_pool, cpu);
+
+		/* If pool is not online, then no entropy is present. */
+		if (!lrng_pcpu_pool_online(cpu))
+			continue;
+
+		/* Obtain entropy statement like for the aux pool */
+		found_irqs = atomic_xchg_relaxed(
+				per_cpu_ptr(&lrng_pcpu_array_irqs, cpu), 0);
+		/* Cap to maximum amount of data we can hold */
+		found_irqs = min_t(u32, found_irqs, digestsize_irqs);
+
+		spin_lock_irqsave(lock, flags2);
+		ret = crypto_cb->lrng_hash_update(shash, pcpu_pool,
+						  LRNG_MAX_DIGESTSIZE);
+		spin_unlock_irqrestore(lock, flags2);
+
+		if (ret)
+			goto err;
+
+		collected_irqs += found_irqs;
+		if (collected_irqs > requested_irqs) {
+			pcpu_unused_irqs = collected_irqs - requested_irqs;
+			atomic_add_return_relaxed(pcpu_unused_irqs,
+				per_cpu_ptr(&lrng_pcpu_array_irqs, cpu));
+			collected_irqs = requested_irqs;
+		}
+		pr_debug("%u interrupts used from entropy pool of CPU %d, %u interrupts remain unused\n",
+			 found_irqs - pcpu_unused_irqs, cpu, pcpu_unused_irqs);
+	}
+
+	ret = crypto_cb->lrng_hash_final(shash, pool->aux_pool);
+	if (ret)
+		goto err;
+
+	read_unlock_irqrestore(&drng->hash_lock, flags);
+
+	collected_ent_bits += lrng_data_to_entropy(collected_irqs);
+
+	/*
+	 * Truncate to available entropy as implicitly allowed by SP800-90B
+	 * section 3.1.5.1.1 table 1 which awards truncated hashes full
+	 * entropy.
+	 *
+	 * During boot time, we read requested_bits data with
+	 * collected_ent_bits entropy. In case our conservative entropy
+	 * estimate underestimates the available entropy we can transport as
+	 * much available entropy as possible. The entropy pool does not
+	 * operate compliant to the German AIS 21/31 NTG.1 yet.
+	 */
+	memcpy(outbuf, pool->aux_pool, fully_seeded ? collected_ent_bits >> 3 :
+						      requested_bits >> 3);
+
+	pr_debug("obtained %u bits of entropy\n", collected_ent_bits);
+
+	return collected_ent_bits;
+
+err:
+	read_unlock_irqrestore(&drng->hash_lock, flags);
+	return 0;
+}
+
+/* Compress the lrng_pcpu_array array into lrng_pcp_pool */
+static inline void lrng_pcpu_array_compress(void)
+{
+	SHASH_DESC_ON_STACK(shash, NULL);
+	struct lrng_drng **lrng_drng = lrng_drng_instances();
+	struct lrng_drng *drng = lrng_drng_init_instance();
+	const struct lrng_crypto_cb *crypto_cb;
+	spinlock_t *lock = this_cpu_ptr(&lrng_pcpu_lock);
+	unsigned long flags, flags2;
+	int node = numa_node_id();
+	void *hash;
+	u8 *pcpu_pool = this_cpu_ptr(lrng_pcpu_pool);
+
+	/* Get NUMA-node local hash instance */
+	if (lrng_drng && lrng_drng[node])
+		drng = lrng_drng[node];
+
+	if (unlikely(!this_cpu_read(lrng_pcpu_lock_init))) {
+		spin_lock_init(lock);
+		this_cpu_write(lrng_pcpu_lock_init, true);
+	}
+
+	read_lock_irqsave(&drng->hash_lock, flags);
+	spin_lock_irqsave(lock, flags2);
+
+	crypto_cb = drng->crypto_cb;
+	hash = drng->hash;
+
+	if (crypto_cb->lrng_hash_init(shash, hash) ||
+	    /* Hash entire per-CPU data array content ... */
+	    crypto_cb->lrng_hash_update(shash,
+					(u8 *)this_cpu_ptr(lrng_pcpu_array),
+					LRNG_DATA_ARRAY_SIZE * sizeof(u32)) ||
+	    /* ... together with per-CPU entropy pool ... */
+	    crypto_cb->lrng_hash_update(shash, pcpu_pool,
+					LRNG_MAX_DIGESTSIZE) ||
+	    /* ... to form new per-CPU entropy pool state. */
+	    crypto_cb->lrng_hash_final(shash, pcpu_pool))
+		pr_warn_ratelimited("Hashing of entropy data failed\n");
+
+	spin_unlock_irqrestore(lock, flags2);
+	read_unlock_irqrestore(&drng->hash_lock, flags);
+}
+
+/* Compress data array into hash */
+static inline void lrng_pcpu_array_to_hash(u32 ptr)
+{
+	u32 *array = this_cpu_ptr(lrng_pcpu_array);
+
+	if (ptr < LRNG_DATA_WORD_MASK)
+		return;
+
+	if (lrng_raw_array_entropy_store(*array)) {
+		u32 i;
+
+		/*
+		 * If we fed even a part of the array to external analysis, we
+		 * mark that the entire array and the per-CPU pool to have no
+		 * entropy. This is due to the non-IID property of the data as
+		 * we do not fully know whether the existing dependencies
+		 * diminish the entropy beyond to what we expect it has.
+		 */
+		atomic_set(this_cpu_ptr(&lrng_pcpu_array_irqs), 0);
+
+		for (i = 1; i < LRNG_DATA_ARRAY_SIZE; i++)
+			lrng_raw_array_entropy_store(*(array + i));
+	} else {
+		lrng_pcpu_array_compress();
+		/* Ping pool handler about received entropy */
+		lrng_pool_add_irq();
+	}
+
+	memset(array, 0, LRNG_DATA_ARRAY_SIZE * sizeof(u32));
+}
+
+/*
+ * Concatenate full 32 bit word at the end of time array even when current
+ * ptr is not aligned to sizeof(data).
+ */
+static inline void _lrng_pcpu_array_add_u32(u32 data)
+{
+	/* Increment pointer by number of slots taken for input value */
+	u32 pre_ptr, mask, ptr = this_cpu_add_return(lrng_pcpu_array_ptr,
+						     LRNG_DATA_SLOTS_PER_UINT);
+
+	/*
+	 * This function injects a unit into the array - guarantee that
+	 * array unit size is equal to data type of input data.
+	 */
+	BUILD_BUG_ON(LRNG_DATA_ARRAY_MEMBER_BITS != (sizeof(data) << 3));
+
+	/*
+	 * The following logic requires at least two units holding
+	 * the data as otherwise the pointer would immediately wrap when
+	 * injection an u32 word.
+	 */
+	BUILD_BUG_ON(LRNG_DATA_NUM_VALUES <= LRNG_DATA_SLOTS_PER_UINT);
+
+	/* ptr to previous unit */
+	pre_ptr = (ptr - LRNG_DATA_SLOTS_PER_UINT) & LRNG_DATA_WORD_MASK;
+	ptr &= LRNG_DATA_WORD_MASK;
+
+	/* mask to split data into the two parts for the two units */
+	mask = ((1 << (pre_ptr & (LRNG_DATA_SLOTS_PER_UINT - 1)) *
+		       LRNG_DATA_SLOTSIZE_BITS)) - 1;
+
+	/* MSB of data go into previous unit */
+	this_cpu_or(lrng_pcpu_array[lrng_data_idx2array(pre_ptr)],
+		    data & ~mask);
+
+	/* Invoke compression as we just filled data array completely */
+	if (unlikely(pre_ptr > ptr))
+		lrng_pcpu_array_to_hash(LRNG_DATA_WORD_MASK);
+
+	/* LSB of data go into current unit */
+	this_cpu_write(lrng_pcpu_array[lrng_data_idx2array(ptr)],
+		       data & mask);
+
+	if (likely(pre_ptr <= ptr))
+		lrng_pcpu_array_to_hash(ptr);
+}
+
+/* Concatenate a 32-bit word at the end of the per-CPU array */
+void lrng_pcpu_array_add_u32(u32 data)
+{
+	_lrng_pcpu_array_add_u32(data);
+}
+
+/* Concatenate data of max LRNG_DATA_SLOTSIZE_MASK at the end of time array */
+static inline void lrng_pcpu_array_add_slot(u32 data)
+{
+	/* Get slot */
+	u32 ptr = this_cpu_inc_return(lrng_pcpu_array_ptr) &
+							LRNG_DATA_WORD_MASK;
+
+	BUILD_BUG_ON(LRNG_DATA_ARRAY_MEMBER_BITS % LRNG_DATA_SLOTSIZE_BITS);
+	/* Ensure consistency of values */
+	BUILD_BUG_ON(LRNG_DATA_ARRAY_MEMBER_BITS !=
+		     sizeof(lrng_pcpu_array[0]) << 3);
+
+	/* Store data into slot */
+	this_cpu_or(lrng_pcpu_array[lrng_data_idx2array(ptr)],
+		    lrng_data_slot_val(data, lrng_data_idx2slot(ptr)));
+
+	lrng_pcpu_array_to_hash(ptr);
+}
+
+/*
+ * Batching up of entropy in per-CPU array before injecting into entropy pool.
+ *
+ * The random32_data is solely to be used for the external random32 PRNG.
+ */
+static inline void lrng_time_process(u32 random32_data)
+{
+	u32 now_time = random_get_entropy();
+	u32 now_time_masked = now_time & LRNG_DATA_SLOTSIZE_MASK;
+	enum lrng_health_res health_test;
+
+	/* During boot time, we process the full time stamp */
+	if (unlikely(!lrng_state_fully_seeded())) {
+
+		/* Seed random32 PRNG with data not used by LRNG. */
+		this_cpu_add(net_rand_state.s1, random32_data);
+
+		if (lrng_raw_hires_entropy_store(now_time))
+			goto out;
+
+		health_test = lrng_health_test(now_time);
+		if (health_test > lrng_health_fail_use)
+			goto out;
+
+		if (health_test == lrng_health_pass)
+			atomic_inc_return(this_cpu_ptr(&lrng_pcpu_array_irqs));
+
+		_lrng_pcpu_array_add_u32(now_time);
+	} else {
+		/* Runtime operation */
+		if (lrng_raw_hires_entropy_store(now_time_masked))
+			goto out;
+
+		/* Seed random32 PRNG with data not used by LRNG. */
+		this_cpu_add(net_rand_state.s1,
+			(now_time & ~LRNG_DATA_SLOTSIZE_MASK) ^ random32_data);
+
+		health_test = lrng_health_test(now_time_masked);
+		if (health_test > lrng_health_fail_use)
+			goto out;
+
+		/* Interrupt delivers entropy if health test passes */
+		if (health_test == lrng_health_pass)
+			atomic_inc_return(this_cpu_ptr(&lrng_pcpu_array_irqs));
+
+		lrng_pcpu_array_add_slot(now_time_masked);
+	}
+
+out:
+	lrng_perf_time(now_time);
+}
+
+/* Hot code path - Callback for interrupt handler */
+void add_interrupt_randomness(int irq, int irq_flg)
+{
+	u32 tmp;
+
+	if (lrng_pool_highres_timer()) {
+		tmp = lrng_raw_irq_entropy_store(irq) ? 0 : irq;
+		tmp ^= lrng_raw_irqflags_entropy_store(irq_flg) ? 0 : irq_flg;
+		tmp ^= lrng_raw_retip_entropy_store(_RET_IP_) ? 0 : _RET_IP_;
+		lrng_time_process(tmp);
+	} else {
+		struct pt_regs *regs = get_irq_regs();
+		static atomic_t reg_idx = ATOMIC_INIT(0);
+		u64 ip;
+
+		if (regs) {
+			u32 *ptr = (u32 *)regs;
+			int reg_ptr = atomic_add_return_relaxed(1, &reg_idx);
+			size_t n = (sizeof(struct pt_regs) / sizeof(u32));
+
+			ip = instruction_pointer(regs);
+			tmp = *(ptr + (reg_ptr % n));
+			tmp = lrng_raw_regs_entropy_store(tmp) ? 0 : tmp;
+			_lrng_pcpu_array_add_u32(tmp);
+		} else {
+			ip = _RET_IP_;
+		}
+
+		lrng_time_process(lrng_raw_retip_entropy_store(ip) ? 0 : ip);
+
+		/*
+		 * The XOR operation combining the different values is not
+		 * considered to destroy entropy since the entirety of all
+		 * processed values delivers the entropy (and not each
+		 * value separately of the other values).
+		 */
+		ip >>= 32;
+		tmp = lrng_raw_jiffies_entropy_store(jiffies) ? 0 : jiffies;
+		tmp ^= lrng_raw_irq_entropy_store(irq) ? 0 : irq;
+		tmp ^= lrng_raw_irqflags_entropy_store(irq_flg) ? 0 : irq_flg;
+		tmp ^= lrng_raw_retip_entropy_store(ip) ? 0 : ip;
+		_lrng_pcpu_array_add_u32(tmp);
+	}
+}
+EXPORT_SYMBOL(add_interrupt_randomness);
diff --git a/drivers/char/lrng/lrng_sw_noise.h b/drivers/char/lrng/lrng_sw_noise.h
new file mode 100644
index 000000000000..ec9d78ba5128
--- /dev/null
+++ b/drivers/char/lrng/lrng_sw_noise.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * LRNG Slow Noise Source: Time stamp array handling
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+/*
+ * To limit the impact on the interrupt handling, the LRNG concatenates
+ * entropic LSB parts of the time stamps in a per-CPU array and only
+ * injects them into the entropy pool when the array is full.
+ */
+
+/* Store multiple integers in one u32 */
+#define LRNG_DATA_SLOTSIZE_BITS		(8)
+#define LRNG_DATA_SLOTSIZE_MASK		((1 << LRNG_DATA_SLOTSIZE_BITS) - 1)
+#define LRNG_DATA_ARRAY_MEMBER_BITS	(4 << 3) /* ((sizeof(u32)) << 3) */
+#define LRNG_DATA_SLOTS_PER_UINT	(LRNG_DATA_ARRAY_MEMBER_BITS / \
+					 LRNG_DATA_SLOTSIZE_BITS)
+
+/*
+ * Number of time values to store in the array - in small environments
+ * only one atomic_t variable per CPU is used.
+ */
+#define LRNG_DATA_NUM_VALUES		(CONFIG_LRNG_COLLECTION_SIZE)
+/* Mask of LSB of time stamp to store */
+#define LRNG_DATA_WORD_MASK		(LRNG_DATA_NUM_VALUES - 1)
+
+#define LRNG_DATA_SLOTS_MASK		(LRNG_DATA_SLOTS_PER_UINT - 1)
+#define LRNG_DATA_ARRAY_SIZE		(LRNG_DATA_NUM_VALUES /	\
+					 LRNG_DATA_SLOTS_PER_UINT)
+
+/* Starting bit index of slot */
+static inline unsigned int lrng_data_slot2bitindex(unsigned int slot)
+{
+	return (LRNG_DATA_SLOTSIZE_BITS * slot);
+}
+
+/* Convert index into the array index */
+static inline unsigned int lrng_data_idx2array(unsigned int idx)
+{
+	return idx / LRNG_DATA_SLOTS_PER_UINT;
+}
+
+/* Convert index into the slot of a given array index */
+static inline unsigned int lrng_data_idx2slot(unsigned int idx)
+{
+	return idx & LRNG_DATA_SLOTS_MASK;
+}
+
+/* Convert value into slot value */
+static inline unsigned int lrng_data_slot_val(unsigned int val,
+					      unsigned int slot)
+{
+	return val << lrng_data_slot2bitindex(slot);
+}
diff --git a/include/linux/lrng.h b/include/linux/lrng.h
new file mode 100644
index 000000000000..3dee5b769236
--- /dev/null
+++ b/include/linux/lrng.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * Copyright (C) 2018 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#ifndef _LRNG_H
+#define _LRNG_H
+
+#include <crypto/hash.h>
+#include <linux/errno.h>
+#include <linux/types.h>
+
+/**
+ * struct lrng_crypto_cb - cryptographic callback functions
+ * @lrng_drng_name		Name of DRNG
+ * @lrng_hash_name		Name of Hash used for reading entropy pool
+ * @lrng_drng_alloc:		Allocate DRNG -- the provided integer should be
+ *				used for sanity checks.
+ *				return: allocated data structure or PTR_ERR on
+ *					error
+ * @lrng_drng_dealloc:		Deallocate DRNG
+ * @lrng_drng_seed_helper:	Seed the DRNG with data of arbitrary length
+ *				drng: is pointer to data structure allocated
+ *				      with lrng_drng_alloc
+ *				return: >= 0 on success, < 0 on error
+ * @lrng_drng_generate_helper:	Generate random numbers from the DRNG with
+ *				arbitrary length
+ * @lrng_hash_alloc:		Allocate the hash for reading the entropy pool
+ *				return: allocated data structure (NULL is
+ *					success too) or ERR_PTR on error
+ * @lrng_hash_dealloc:		Deallocate Hash
+ * @lrng_hash_digestsize:	Return the digestsize for the used hash to read
+ *				out entropy pool
+ *				hash: is pointer to data structure allocated
+ *				      with lrng_hash_alloc
+ *				return: size of digest of hash in bytes
+ * @lrng_hash_init:		Initialize hash
+ *				hash: is pointer to data structure allocated
+ *				      with lrng_hash_alloc
+ *				return: 0 on success, < 0 on error
+ * @lrng_hash_update:		Update hash operation
+ *				hash: is pointer to data structure allocated
+ *				      with lrng_hash_alloc
+ *				return: 0 on success, < 0 on error
+ * @lrng_hash_final		Final hash operation
+ *				hash: is pointer to data structure allocated
+ *				      with lrng_hash_alloc
+ *				return: 0 on success, < 0 on error
+ *
+ * Assumptions:
+ *
+ * 1. Hash operation will not sleep
+ * 2. The hash' volatile state information is provided with *shash by caller.
+ */
+struct lrng_crypto_cb {
+	const char *(*lrng_drng_name)(void);
+	const char *(*lrng_hash_name)(void);
+	void *(*lrng_drng_alloc)(u32 sec_strength);
+	void (*lrng_drng_dealloc)(void *drng);
+	int (*lrng_drng_seed_helper)(void *drng, const u8 *inbuf, u32 inbuflen);
+	int (*lrng_drng_generate_helper)(void *drng, u8 *outbuf, u32 outbuflen);
+	void *(*lrng_hash_alloc)(void);
+	void (*lrng_hash_dealloc)(void *hash);
+	u32 (*lrng_hash_digestsize)(void *hash);
+	int (*lrng_hash_init)(struct shash_desc *shash, void *hash);
+	int (*lrng_hash_update)(struct shash_desc *shash, const u8 *inbuf,
+				u32 inbuflen);
+	int (*lrng_hash_final)(struct shash_desc *shash, u8 *digest);
+};
+
+/* Register cryptographic backend */
+#ifdef CONFIG_LRNG_DRNG_SWITCH
+int lrng_set_drng_cb(const struct lrng_crypto_cb *cb);
+#else	/* CONFIG_LRNG_DRNG_SWITCH */
+static inline int
+lrng_set_drng_cb(const struct lrng_crypto_cb *cb) { return -EOPNOTSUPP; }
+#endif	/* CONFIG_LRNG_DRNG_SWITCH */
+
+#endif /* _LRNG_H */
-- 
2.26.2

[PATCH v36 02/13] LRNG - allocate one DRNG instance per NUMA node

[Stephan Müller]

In order to improve NUMA-locality when serving getrandom(2) requests,
allocate one DRNG instance per node.

The DRNG instance that is present right from the start of the kernel is
reused as the first per-NUMA-node DRNG. For all remaining online NUMA
nodes a new DRNG instance is allocated.

During boot time, the multiple DRNG instances are seeded sequentially.
With this, the first DRNG instance (referenced as the initial DRNG
in the code) is completely seeded with 256 bits of entropy before the
next DRNG instance is completely seeded.

When random numbers are requested, the NUMA-node-local DRNG is checked
whether it has been already fully seeded. If this is not the case, the
initial DRNG is used to serve the request.

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
CC: Eric Biggers <ebiggers@kernel.org>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 drivers/char/lrng/Makefile        |   2 +
 drivers/char/lrng/lrng_internal.h |   5 ++
 drivers/char/lrng/lrng_numa.c     | 108 ++++++++++++++++++++++++++++++
 3 files changed, 115 insertions(+)
 create mode 100644 drivers/char/lrng/lrng_numa.c

diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index e72e01c15bb9..29724c65287d 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -7,3 +7,5 @@ obj-y				+= lrng_pool.o lrng_aux.o \
 				   lrng_sw_noise.o lrng_archrandom.o \
 				   lrng_drng.o lrng_chacha20.o \
 				   lrng_interfaces.o
+
+obj-$(CONFIG_NUMA)		+= lrng_numa.o
diff --git a/drivers/char/lrng/lrng_internal.h b/drivers/char/lrng/lrng_internal.h
index f9b80ce42341..f858effcf710 100644
--- a/drivers/char/lrng/lrng_internal.h
+++ b/drivers/char/lrng/lrng_internal.h
@@ -219,8 +219,13 @@ int lrng_drng_get_sleep(u8 *outbuf, u32 outbuflen);
 void lrng_drng_force_reseed(void);
 void lrng_drng_seed_work(struct work_struct *dummy);
 
+#ifdef CONFIG_NUMA
+struct lrng_drng **lrng_drng_instances(void);
+void lrng_drngs_numa_alloc(void);
+#else	/* CONFIG_NUMA */
 static inline struct lrng_drng **lrng_drng_instances(void) { return NULL; }
 static inline void lrng_drngs_numa_alloc(void) { return; }
+#endif /* CONFIG_NUMA */
 
 /************************** Entropy pool management ***************************/
 
diff --git a/drivers/char/lrng/lrng_numa.c b/drivers/char/lrng/lrng_numa.c
new file mode 100644
index 000000000000..7f1f0dade1b6
--- /dev/null
+++ b/drivers/char/lrng/lrng_numa.c
@@ -0,0 +1,108 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG NUMA support
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/lrng.h>
+#include <linux/slab.h>
+
+#include "lrng_internal.h"
+
+static struct lrng_drng **lrng_drng __read_mostly = NULL;
+
+struct lrng_drng **lrng_drng_instances(void)
+{
+	return smp_load_acquire(&lrng_drng);
+}
+
+/* Allocate the data structures for the per-NUMA node DRNGs */
+static void _lrng_drngs_numa_alloc(struct work_struct *work)
+{
+	struct lrng_drng **drngs;
+	struct lrng_drng *lrng_drng_init = lrng_drng_init_instance();
+	u32 node;
+	bool init_drng_used = false;
+
+	mutex_lock(&lrng_crypto_cb_update);
+
+	/* per-NUMA-node DRNGs are already present */
+	if (lrng_drng)
+		goto unlock;
+
+	drngs = kcalloc(nr_node_ids, sizeof(void *), GFP_KERNEL|__GFP_NOFAIL);
+	for_each_online_node(node) {
+		struct lrng_drng *drng;
+
+		if (!init_drng_used) {
+			drngs[node] = lrng_drng_init;
+			init_drng_used = true;
+			continue;
+		}
+
+		drng = kmalloc_node(sizeof(struct lrng_drng),
+				     GFP_KERNEL|__GFP_NOFAIL, node);
+		memset(drng, 0, sizeof(lrng_drng));
+
+		drng->crypto_cb = lrng_drng_init->crypto_cb;
+		drng->drng = drng->crypto_cb->lrng_drng_alloc(
+					LRNG_DRNG_SECURITY_STRENGTH_BYTES);
+		if (IS_ERR(drng->drng)) {
+			kfree(drng);
+			goto err;
+		}
+
+		drng->hash = drng->crypto_cb->lrng_hash_alloc();
+		if (IS_ERR(drng->hash)) {
+			drng->crypto_cb->lrng_drng_dealloc(drng->drng);
+			kfree(drng);
+			goto err;
+		}
+
+		mutex_init(&drng->lock);
+		spin_lock_init(&drng->spin_lock);
+		rwlock_init(&drng->hash_lock);
+
+		/*
+		 * No reseeding of NUMA DRNGs from previous DRNGs as this
+		 * would complicate the code. Let it simply reseed.
+		 */
+		lrng_drng_reset(drng);
+		drngs[node] = drng;
+
+		lrng_pool_inc_numa_node();
+		pr_info("DRNG and entropy pool read hash for NUMA node %d allocated\n",
+			node);
+	}
+
+	/* counterpart to smp_load_acquire in lrng_drng_instances */
+	if (!cmpxchg_release(&lrng_drng, NULL, drngs))
+		goto unlock;
+
+err:
+	for_each_online_node(node) {
+		struct lrng_drng *drng = drngs[node];
+
+		if (drng == lrng_drng_init)
+			continue;
+
+		if (drng) {
+			drng->crypto_cb->lrng_drng_dealloc(drng->drng);
+			kfree(drng);
+		}
+	}
+	kfree(drngs);
+
+unlock:
+	mutex_unlock(&lrng_crypto_cb_update);
+}
+
+static DECLARE_WORK(lrng_drngs_numa_alloc_work, _lrng_drngs_numa_alloc);
+
+void lrng_drngs_numa_alloc(void)
+{
+	schedule_work(&lrng_drngs_numa_alloc_work);
+}
-- 
2.26.2

[PATCH v36 03/13] LRNG - sysctls and /proc interface

[Stephan Müller]

The LRNG sysctl interface provides the same controls as the existing
/dev/random implementation. These sysctls behave identically and are
implemented identically. The goal is to allow a possible merge of the
existing /dev/random implementation with this implementation which
implies that this patch tries have a very close similarity. Yet, all
sysctls are documented at [1].

In addition, it provides the file lrng_type which provides details about
the LRNG:

- the name of the DRNG that produces the random numbers for /dev/random,
/dev/urandom, getrandom(2)

- the hash used to produce random numbers from the entropy pool

- the number of secondary DRNG instances

- indicator whether the LRNG operates SP800-90B compliant

- indicator whether a high-resolution timer is identified - only with a
high-resolution timer the interrupt noise source will deliver sufficient
entropy

- indicator whether the LRNG has been minimally seeded (i.e. is the
secondary DRNG seeded with at least 128 bits of of entropy)

- indicator whether the LRNG has been fully seeded (i.e. is the
secondary DRNG seeded with at least 256 bits of entropy)

[1] https://www.chronox.de/lrng.html

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 drivers/char/lrng/Makefile          |   1 +
 drivers/char/lrng/lrng_interfaces.c |   2 -
 drivers/char/lrng/lrng_internal.h   |   4 +
 drivers/char/lrng/lrng_proc.c       | 182 ++++++++++++++++++++++++++++
 4 files changed, 187 insertions(+), 2 deletions(-)
 create mode 100644 drivers/char/lrng/lrng_proc.c

diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 29724c65287d..ac97f0b11cb7 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -9,3 +9,4 @@ obj-y				+= lrng_pool.o lrng_aux.o \
 				   lrng_interfaces.o
 
 obj-$(CONFIG_NUMA)		+= lrng_numa.o
+obj-$(CONFIG_SYSCTL)		+= lrng_proc.o
diff --git a/drivers/char/lrng/lrng_interfaces.c b/drivers/char/lrng/lrng_interfaces.c
index 19d01d3f7492..b55de97523ad 100644
--- a/drivers/char/lrng/lrng_interfaces.c
+++ b/drivers/char/lrng/lrng_interfaces.c
@@ -38,8 +38,6 @@ static DECLARE_WAIT_QUEUE_HEAD(lrng_write_wait);
 static DECLARE_WAIT_QUEUE_HEAD(lrng_init_wait);
 static struct fasync_struct *fasync;
 
-struct ctl_table random_table[];
-
 /********************************** Helper ***********************************/
 
 /* Is the DRNG seed level too low? */
diff --git a/drivers/char/lrng/lrng_internal.h b/drivers/char/lrng/lrng_internal.h
index f858effcf710..2f8c14ffbaf3 100644
--- a/drivers/char/lrng/lrng_internal.h
+++ b/drivers/char/lrng/lrng_internal.h
@@ -113,7 +113,11 @@ void lrng_cc20_init_state_boot(struct chacha20_state *state);
 
 /********************************** /proc *************************************/
 
+#ifdef CONFIG_SYSCTL
+void lrng_pool_inc_numa_node(void);
+#else
 static inline void lrng_pool_inc_numa_node(void) { }
+#endif
 
 /****************************** LRNG interfaces *******************************/
 
diff --git a/drivers/char/lrng/lrng_proc.c b/drivers/char/lrng/lrng_proc.c
new file mode 100644
index 000000000000..b2985a2ae0f7
--- /dev/null
+++ b/drivers/char/lrng/lrng_proc.c
@@ -0,0 +1,182 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG proc and sysctl interfaces
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#include <linux/lrng.h>
+#include <linux/proc_fs.h>
+#include <linux/seq_file.h>
+#include <linux/sysctl.h>
+#include <linux/uuid.h>
+
+#include "lrng_internal.h"
+#include "lrng_sw_noise.h"
+
+/*
+ * This function is used to return both the bootid UUID, and random
+ * UUID.  The difference is in whether table->data is NULL; if it is,
+ * then a new UUID is generated and returned to the user.
+ *
+ * If the user accesses this via the proc interface, the UUID will be
+ * returned as an ASCII string in the standard UUID format; if via the
+ * sysctl system call, as 16 bytes of binary data.
+ */
+static int lrng_proc_do_uuid(struct ctl_table *table, int write,
+			     void *buffer, size_t *lenp, loff_t *ppos)
+{
+	struct ctl_table fake_table;
+	unsigned char buf[64], tmp_uuid[16], *uuid;
+
+	uuid = table->data;
+	if (!uuid) {
+		uuid = tmp_uuid;
+		generate_random_uuid(uuid);
+	} else {
+		static DEFINE_SPINLOCK(bootid_spinlock);
+
+		spin_lock(&bootid_spinlock);
+		if (!uuid[8])
+			generate_random_uuid(uuid);
+		spin_unlock(&bootid_spinlock);
+	}
+
+	sprintf(buf, "%pU", uuid);
+
+	fake_table.data = buf;
+	fake_table.maxlen = sizeof(buf);
+
+	return proc_dostring(&fake_table, write, buffer, lenp, ppos);
+}
+
+static int lrng_proc_do_entropy(struct ctl_table *table, int write,
+				void *buffer, size_t *lenp, loff_t *ppos)
+{
+	struct ctl_table fake_table;
+	int entropy_count;
+
+	entropy_count = lrng_avail_entropy();
+
+	fake_table.data = &entropy_count;
+	fake_table.maxlen = sizeof(entropy_count);
+
+	return proc_dointvec(&fake_table, write, buffer, lenp, ppos);
+}
+
+static int lrng_proc_do_poolsize(struct ctl_table *table, int write,
+				 void *buffer, size_t *lenp, loff_t *ppos)
+{
+	struct ctl_table fake_table;
+	int entropy_count;
+
+	/* LRNG can at most retain entropy in per-CPU pools and aux pool */
+	entropy_count = lrng_get_digestsize() * (num_online_cpus() + 1);
+
+	fake_table.data = &entropy_count;
+	fake_table.maxlen = sizeof(entropy_count);
+
+	return proc_dointvec(&fake_table, write, buffer, lenp, ppos);
+}
+
+static int lrng_min_write_thresh;
+static int lrng_max_write_thresh = LRNG_MAX_DIGESTSIZE;
+static char lrng_sysctl_bootid[16];
+static int lrng_drng_reseed_max_min;
+
+struct ctl_table random_table[] = {
+	{
+		.procname	= "poolsize",
+		.maxlen		= sizeof(int),
+		.mode		= 0444,
+		.proc_handler	= lrng_proc_do_poolsize,
+	},
+	{
+		.procname	= "entropy_avail",
+		.maxlen		= sizeof(int),
+		.mode		= 0444,
+		.proc_handler	= lrng_proc_do_entropy,
+	},
+	{
+		.procname	= "write_wakeup_threshold",
+		.data		= &lrng_write_wakeup_bits,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec_minmax,
+		.extra1		= &lrng_min_write_thresh,
+		.extra2		= &lrng_max_write_thresh,
+	},
+	{
+		.procname	= "boot_id",
+		.data		= &lrng_sysctl_bootid,
+		.maxlen		= 16,
+		.mode		= 0444,
+		.proc_handler	= lrng_proc_do_uuid,
+	},
+	{
+		.procname	= "uuid",
+		.maxlen		= 16,
+		.mode		= 0444,
+		.proc_handler	= lrng_proc_do_uuid,
+	},
+	{
+		.procname       = "urandom_min_reseed_secs",
+		.data           = &lrng_drng_reseed_max_time,
+		.maxlen         = sizeof(int),
+		.mode           = 0644,
+		.proc_handler   = proc_dointvec,
+		.extra1		= &lrng_drng_reseed_max_min,
+	},
+	{ }
+};
+
+/* Number of online DRNGs */
+static u32 numa_drngs = 1;
+
+void lrng_pool_inc_numa_node(void)
+{
+	numa_drngs++;
+}
+
+static int lrng_proc_type_show(struct seq_file *m, void *v)
+{
+	struct lrng_drng *lrng_drng_init = lrng_drng_init_instance();
+	unsigned long flags = 0;
+	unsigned char buf[350];
+
+	lrng_drng_lock(lrng_drng_init, &flags);
+	snprintf(buf, sizeof(buf),
+		 "DRNG name: %s\n"
+		 "Hash for reading entropy pool: %s\n"
+		 "Hash for operating aux entropy pool: %s\n"
+		 "LRNG security strength in bits: %d\n"
+		 "per-CPU interrupt collection size: %u\n"
+		 "number of DRNG instances: %u\n"
+		 "SP800-90B compliance: %s\n"
+		 "High-resolution timer: %s\n"
+		 "LRNG minimally seeded: %s\n"
+		 "LRNG fully seeded: %s\n",
+		 lrng_drng_init->crypto_cb->lrng_drng_name(),
+		 lrng_drng_init->crypto_cb->lrng_hash_name(),
+		 lrng_drng_init->crypto_cb->lrng_hash_name(),
+		 lrng_security_strength(),
+		 LRNG_DATA_NUM_VALUES,
+		 numa_drngs,
+		 lrng_sp80090b_compliant() ? "true" : "false",
+		 lrng_pool_highres_timer() ? "true" : "false",
+		 lrng_state_min_seeded() ? "true" : "false",
+		 lrng_state_fully_seeded() ? "true" : "false");
+	lrng_drng_unlock(lrng_drng_init, &flags);
+
+	seq_write(m, buf, strlen(buf));
+
+	return 0;
+}
+
+static int __init lrng_proc_type_init(void)
+{
+	proc_create_single("lrng_type", 0444, NULL, &lrng_proc_type_show);
+	return 0;
+}
+
+module_init(lrng_proc_type_init);
-- 
2.26.2

[PATCH v36 04/13] LRNG - add switchable DRNG support

[Stephan Müller]

The DRNG switch support allows replacing the DRNG mechanism of the
LRNG. The switching support rests on the interface definition of
include/linux/lrng.h. A new DRNG is implemented by filling in the
interface defined in this header file.

In addition to the DRNG, the extension also has to provide a hash
implementation that is used to hash the entropy pool for random number
extraction.

Note: It is permissible to implement a DRNG whose operations may sleep.
However, the hash function must not sleep.

The switchable DRNG support allows replacing the DRNG at runtime.
However, only one DRNG extension is allowed to be loaded at any given
time. Before replacing it with another DRNG implementation, the possibly
existing DRNG extension must be unloaded.

The switchable DRNG extension activates the new DRNG during load time.
It is expected, however, that such a DRNG switch would be done only once
by an administrator to load the intended DRNG implementation.

It is permissible to compile DRNG extensions either as kernel modules or
statically. The initialization of the DRNG extension should be performed
with a late_initcall to ensure the extension is available when user
space starts but after all other initialization completed.
The initialization is performed by registering the function call data
structure with the lrng_set_drng_cb function. In order to unload the
DRNG extension, lrng_set_drng_cb must be invoked with the NULL
parameter.

The DRNG extension should always provide a security strength that is at
least as strong as LRNG_DRNG_SECURITY_STRENGTH_BITS.

The hash extension must not sleep and must not maintain a separate
state.

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 drivers/char/lrng/Kconfig       |   7 ++
 drivers/char/lrng/Makefile      |   1 +
 drivers/char/lrng/lrng_switch.c | 203 ++++++++++++++++++++++++++++++++
 3 files changed, 211 insertions(+)
 create mode 100644 drivers/char/lrng/lrng_switch.c

diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index fbbcf2ef43b6..e211fcf5aa8b 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -69,4 +69,11 @@ config LRNG_COLLECTION_SIZE
 	default 512 if LRNG_COLLECTION_SIZE_512
 	default 1024 if LRNG_COLLECTION_SIZE_1024
 
+menuconfig LRNG_DRNG_SWITCH
+	bool "Support DRNG runtime switching"
+	help
+	  The Linux RNG per default uses a ChaCha20 DRNG that is
+	  accessible via the external interfaces. With this configuration
+	  option other DRNGs can be selected and loaded at runtime.
+
 endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index ac97f0b11cb7..0eb4a6849c88 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -10,3 +10,4 @@ obj-y				+= lrng_pool.o lrng_aux.o \
 
 obj-$(CONFIG_NUMA)		+= lrng_numa.o
 obj-$(CONFIG_SYSCTL)		+= lrng_proc.o
+obj-$(CONFIG_LRNG_DRNG_SWITCH)	+= lrng_switch.o
diff --git a/drivers/char/lrng/lrng_switch.c b/drivers/char/lrng/lrng_switch.c
new file mode 100644
index 000000000000..cbaf5cd544aa
--- /dev/null
+++ b/drivers/char/lrng/lrng_switch.c
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG DRNG switching support
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/lrng.h>
+
+#include "lrng_internal.h"
+
+static int lrng_drng_switch(struct lrng_drng *drng_store,
+			    const struct lrng_crypto_cb *cb, int node)
+{
+	const struct lrng_crypto_cb *old_cb;
+	unsigned long flags = 0, flags2 = 0;
+	int ret;
+	u8 seed[LRNG_DRNG_SECURITY_STRENGTH_BYTES];
+	void *new_drng = cb->lrng_drng_alloc(LRNG_DRNG_SECURITY_STRENGTH_BYTES);
+	void *old_drng, *new_hash, *old_hash;
+	u32 current_security_strength;
+	bool sl = false, reset_drng = !lrng_get_available();
+
+	if (IS_ERR(new_drng)) {
+		pr_warn("could not allocate new DRNG for NUMA node %d (%ld)\n",
+			node, PTR_ERR(new_drng));
+		return PTR_ERR(new_drng);
+	}
+
+	new_hash = cb->lrng_hash_alloc();
+	if (IS_ERR(new_hash)) {
+		pr_warn("could not allocate new LRNG pool hash (%ld)\n",
+			PTR_ERR(new_hash));
+		cb->lrng_drng_dealloc(new_drng);
+		return PTR_ERR(new_hash);
+	}
+
+	if (cb->lrng_hash_digestsize(new_hash) > LRNG_MAX_DIGESTSIZE) {
+		pr_warn("digest size of newly requested hash too large\n");
+		cb->lrng_hash_dealloc(new_hash);
+		cb->lrng_drng_dealloc(new_drng);
+		return -EINVAL;
+	}
+
+	current_security_strength = lrng_security_strength();
+	lrng_drng_lock(drng_store, &flags);
+
+	/*
+	 * Pull from existing DRNG to seed new DRNG regardless of seed status
+	 * of old DRNG -- the entropy state for the DRNG is left unchanged which
+	 * implies that als the new DRNG is reseeded when deemed necessary. This
+	 * seeding of the new DRNG shall only ensure that the new DRNG has the
+	 * same entropy as the old DRNG.
+	 */
+	ret = drng_store->crypto_cb->lrng_drng_generate_helper(
+				drng_store->drng, seed, sizeof(seed));
+	lrng_drng_unlock(drng_store, &flags);
+
+	if (ret < 0) {
+		reset_drng = true;
+		pr_warn("getting random data from DRNG failed for NUMA node %d (%d)\n",
+			node, ret);
+	} else {
+		/* seed new DRNG with data */
+		ret = cb->lrng_drng_seed_helper(new_drng, seed, ret);
+		if (ret < 0) {
+			reset_drng = true;
+			pr_warn("seeding of new DRNG failed for NUMA node %d (%d)\n",
+				node, ret);
+		} else {
+			pr_debug("seeded new DRNG of NUMA node %d instance from old DRNG instance\n",
+				 node);
+		}
+	}
+
+	mutex_lock(&drng_store->lock);
+	write_lock_irqsave(&drng_store->hash_lock, flags2);
+	/*
+	 * If we switch the DRNG from the initial ChaCha20 DRNG to something
+	 * else, there is a lock transition from spin lock to mutex (see
+	 * lrng_drng_is_atomic and how the lock is taken in lrng_drng_lock).
+	 * Thus, we need to take both locks during the transition phase.
+	 */
+	if (lrng_drng_is_atomic(drng_store)) {
+		spin_lock_irqsave(&drng_store->spin_lock, flags);
+		sl = true;
+	} else {
+		__acquire(&drng_store->spin_lock);
+	}
+
+	if (reset_drng)
+		lrng_drng_reset(drng_store);
+
+	old_drng = drng_store->drng;
+	old_cb = drng_store->crypto_cb;
+	drng_store->drng = new_drng;
+	drng_store->crypto_cb = cb;
+
+	old_hash = drng_store->hash;
+	drng_store->hash = new_hash;
+	pr_info("Entropy pool read-hash allocated for DRNG for NUMA node %d\n",
+		node);
+
+	lrng_set_digestsize(cb->lrng_hash_digestsize(new_hash));
+
+	/* Reseed if previous LRNG security strength was insufficient */
+	if (current_security_strength < lrng_security_strength())
+		drng_store->force_reseed = true;
+
+	if (sl)
+		spin_unlock_irqrestore(&drng_store->spin_lock, flags);
+	else
+		__release(&drng_store->spin_lock);
+	write_unlock_irqrestore(&drng_store->hash_lock, flags2);
+	mutex_unlock(&drng_store->lock);
+
+	/* ChaCha20 serves as atomic instance left untouched. */
+	if (old_drng != &chacha20) {
+		old_cb->lrng_drng_dealloc(old_drng);
+		old_cb->lrng_hash_dealloc(old_hash);
+	}
+
+	pr_info("DRNG of NUMA node %d switched\n", node);
+
+	return 0;
+}
+
+/*
+ * Switch the existing DRNG instances with new using the new crypto callbacks.
+ * The caller must hold the lrng_crypto_cb_update lock.
+ */
+static int lrng_drngs_switch(const struct lrng_crypto_cb *cb)
+{
+	struct lrng_drng **lrng_drng = lrng_drng_instances();
+	struct lrng_drng *lrng_drng_init = lrng_drng_init_instance();
+	int ret = 0;
+
+	/* Update DRNG */
+	if (lrng_drng) {
+		u32 node;
+
+		for_each_online_node(node) {
+			if (lrng_drng[node])
+				ret = lrng_drng_switch(lrng_drng[node], cb,
+						       node);
+		}
+	} else {
+		ret = lrng_drng_switch(lrng_drng_init, cb, 0);
+	}
+
+	if (!ret)
+		lrng_set_available();
+
+	return 0;
+}
+
+/**
+ * lrng_set_drng_cb - Register new cryptographic callback functions for DRNG
+ * The registering implies that all old DRNG states are replaced with new
+ * DRNG states.
+ *
+ * @cb: Callback functions to be registered -- if NULL, use the default
+ *	callbacks pointing to the ChaCha20 DRNG.
+ *
+ * Return:
+ * * 0 on success
+ * * < 0 on error
+ */
+int lrng_set_drng_cb(const struct lrng_crypto_cb *cb)
+{
+	struct lrng_drng *lrng_drng_init = lrng_drng_init_instance();
+	int ret;
+
+	if (!cb)
+		cb = &lrng_cc20_crypto_cb;
+
+	mutex_lock(&lrng_crypto_cb_update);
+
+	/*
+	 * If a callback other than the default is set, allow it only to be
+	 * set back to the default callback. This ensures that multiple
+	 * different callbacks can be registered at the same time. If a
+	 * callback different from the current callback and the default
+	 * callback shall be set, the current callback must be deregistered
+	 * (e.g. the kernel module providing it must be unloaded) and the new
+	 * implementation can be registered.
+	 */
+	if ((cb != &lrng_cc20_crypto_cb) &&
+	    (lrng_drng_init->crypto_cb != &lrng_cc20_crypto_cb)) {
+		pr_warn("disallow setting new cipher callbacks, unload the old callbacks first!\n");
+		ret = -EINVAL;
+		goto out;
+	}
+
+	ret = lrng_drngs_switch(cb);
+
+out:
+	mutex_unlock(&lrng_crypto_cb_update);
+	return ret;
+}
+EXPORT_SYMBOL(lrng_set_drng_cb);
-- 
2.26.2

[PATCH v36 05/13] LRNG - add common generic hash support

[Stephan Müller]

The LRNG switchable DRNG support also allows the replacement of the hash
implementation used as conditioning component. The common generic hash
support code provides the required callbacks using the synchronous hash
implementations of the kernel crypto API.

All synchronous hash implementations supported by the kernel crypto API
can be used as part of the LRNG with this generic support.

The generic support is intended to be configured by separate switchable
DRNG backends.

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
CC: "Peter, Matthias" <matthias.peter@bsi.bund.de>
CC: Roman Drahtmueller <draht@schaltsekun.de>
CC: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
CC: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 drivers/char/lrng/Kconfig           |  7 +++
 drivers/char/lrng/Makefile          |  1 +
 drivers/char/lrng/lrng_kcapi_hash.c | 97 +++++++++++++++++++++++++++++
 drivers/char/lrng/lrng_kcapi_hash.h | 19 ++++++
 4 files changed, 124 insertions(+)
 create mode 100644 drivers/char/lrng/lrng_kcapi_hash.c
 create mode 100644 drivers/char/lrng/lrng_kcapi_hash.h

diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index e211fcf5aa8b..daa2057248ac 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -76,4 +76,11 @@ menuconfig LRNG_DRNG_SWITCH
 	  accessible via the external interfaces. With this configuration
 	  option other DRNGs can be selected and loaded at runtime.
 
+if LRNG_DRNG_SWITCH
+
+config LRNG_KCAPI_HASH
+	bool
+
+endif # LRNG_DRNG_SWITCH
+
 endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 0eb4a6849c88..40f8826edeeb 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -11,3 +11,4 @@ obj-y				+= lrng_pool.o lrng_aux.o \
 obj-$(CONFIG_NUMA)		+= lrng_numa.o
 obj-$(CONFIG_SYSCTL)		+= lrng_proc.o
 obj-$(CONFIG_LRNG_DRNG_SWITCH)	+= lrng_switch.o
+obj-$(CONFIG_LRNG_KCAPI_HASH)	+= lrng_kcapi_hash.o
diff --git a/drivers/char/lrng/lrng_kcapi_hash.c b/drivers/char/lrng/lrng_kcapi_hash.c
new file mode 100644
index 000000000000..c5ddc71f5f8c
--- /dev/null
+++ b/drivers/char/lrng/lrng_kcapi_hash.c
@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Backend for providing the hash primitive using the kernel crypto API.
+ *
+ * Copyright (C) 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <crypto/hash.h>
+
+#include "lrng_kcapi_hash.h"
+
+struct lrng_hash_info {
+	struct crypto_shash *tfm;
+};
+
+static inline void _lrng_kcapi_hash_free(struct lrng_hash_info *lrng_hash)
+{
+	struct crypto_shash *tfm = lrng_hash->tfm;
+
+	crypto_free_shash(tfm);
+	kfree(lrng_hash);
+}
+
+void *lrng_kcapi_hash_alloc(const char *name)
+{
+	struct lrng_hash_info *lrng_hash;
+	struct crypto_shash *tfm;
+	int ret;
+
+	if (!name) {
+		pr_err("Hash name missing\n");
+		return ERR_PTR(-EINVAL);
+	}
+
+	tfm = crypto_alloc_shash(name, 0, 0);
+	if (IS_ERR(tfm)) {
+		pr_err("could not allocate hash %s\n", name);
+		return ERR_CAST(tfm);
+	}
+
+	ret = sizeof(struct lrng_hash_info);
+	lrng_hash = kmalloc(ret, GFP_KERNEL);
+	if (!lrng_hash) {
+		crypto_free_shash(tfm);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	lrng_hash->tfm = tfm;
+
+	pr_info("Hash %s allocated\n", name);
+
+	return lrng_hash;
+}
+EXPORT_SYMBOL(lrng_kcapi_hash_alloc);
+
+u32 lrng_kcapi_hash_digestsize(void *hash)
+{
+	struct lrng_hash_info *lrng_hash = (struct lrng_hash_info *)hash;
+	struct crypto_shash *tfm = lrng_hash->tfm;
+
+	return crypto_shash_digestsize(tfm);
+}
+EXPORT_SYMBOL(lrng_kcapi_hash_digestsize);
+
+void lrng_kcapi_hash_dealloc(void *hash)
+{
+	struct lrng_hash_info *lrng_hash = (struct lrng_hash_info *)hash;
+
+	_lrng_kcapi_hash_free(lrng_hash);
+	pr_info("Hash deallocated\n");
+}
+EXPORT_SYMBOL(lrng_kcapi_hash_dealloc);
+
+int lrng_kcapi_hash_init(struct shash_desc *shash, void *hash)
+{
+	struct lrng_hash_info *lrng_hash = (struct lrng_hash_info *)hash;
+	struct crypto_shash *tfm = lrng_hash->tfm;
+
+	shash->tfm = tfm;
+	return crypto_shash_init(shash);
+}
+EXPORT_SYMBOL(lrng_kcapi_hash_init);
+
+int lrng_kcapi_hash_update(struct shash_desc *shash, const u8 *inbuf,
+			   u32 inbuflen)
+{
+	return crypto_shash_update(shash, inbuf, inbuflen);
+}
+EXPORT_SYMBOL(lrng_kcapi_hash_update);
+
+int lrng_kcapi_hash_final(struct shash_desc *shash, u8 *digest)
+{
+	return crypto_shash_final(shash, digest);
+}
+EXPORT_SYMBOL(lrng_kcapi_hash_final);
diff --git a/drivers/char/lrng/lrng_kcapi_hash.h b/drivers/char/lrng/lrng_kcapi_hash.h
new file mode 100644
index 000000000000..5bca7aba197f
--- /dev/null
+++ b/drivers/char/lrng/lrng_kcapi_hash.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * Copyright (C) 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#ifndef _LRNG_KCAPI_HASH_H
+#define _LRNG_KCAPI_HASH_H
+
+#include <linux/module.h>
+
+void *lrng_kcapi_hash_alloc(const char *name);
+u32 lrng_kcapi_hash_digestsize(void *hash);
+void lrng_kcapi_hash_dealloc(void *hash);
+int lrng_kcapi_hash_init(struct shash_desc *shash, void *hash);
+int lrng_kcapi_hash_update(struct shash_desc *shash, const u8 *inbuf,
+			   u32 inbuflen);
+int lrng_kcapi_hash_final(struct shash_desc *shash, u8 *digest);
+
+#endif /* _LRNG_KCAPI_HASH_H */
-- 
2.26.2

[PATCH v36 06/13] crypto: DRBG - externalize DRBG functions for LRNG

[Stephan Müller]

This patch allows several DRBG functions to be called by the LRNG kernel
code paths outside the drbg.c file.

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 crypto/drbg.c         | 16 ++++++++++------
 include/crypto/drbg.h |  7 +++++++
 2 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/crypto/drbg.c b/crypto/drbg.c
index 3132967a1749..58b1de903def 100644
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -113,7 +113,7 @@
  * the SHA256 / AES 256 over other ciphers. Thus, the favored
  * DRBGs are the latest entries in this array.
  */
-static const struct drbg_core drbg_cores[] = {
+const struct drbg_core drbg_cores[] = {
 #ifdef CONFIG_CRYPTO_DRBG_CTR
 	{
 		.flags = DRBG_CTR | DRBG_STRENGTH128,
@@ -190,6 +190,7 @@ static const struct drbg_core drbg_cores[] = {
 	},
 #endif /* CONFIG_CRYPTO_DRBG_HMAC */
 };
+EXPORT_SYMBOL(drbg_cores);
 
 static int drbg_uninstantiate(struct drbg_state *drbg);
 
@@ -205,7 +206,7 @@ static int drbg_uninstantiate(struct drbg_state *drbg);
  * Return: normalized strength in *bytes* value or 32 as default
  *	   to counter programming errors
  */
-static inline unsigned short drbg_sec_strength(drbg_flag_t flags)
+unsigned short drbg_sec_strength(drbg_flag_t flags)
 {
 	switch (flags & DRBG_STRENGTH_MASK) {
 	case DRBG_STRENGTH128:
@@ -218,6 +219,7 @@ static inline unsigned short drbg_sec_strength(drbg_flag_t flags)
 		return 32;
 	}
 }
+EXPORT_SYMBOL(drbg_sec_strength);
 
 /*
  * FIPS 140-2 continuous self test for the noise source
@@ -1214,7 +1216,7 @@ static int drbg_seed(struct drbg_state *drbg, struct drbg_string *pers,
 }
 
 /* Free all substructures in a DRBG state without the DRBG state structure */
-static inline void drbg_dealloc_state(struct drbg_state *drbg)
+void drbg_dealloc_state(struct drbg_state *drbg)
 {
 	if (!drbg)
 		return;
@@ -1235,12 +1237,13 @@ static inline void drbg_dealloc_state(struct drbg_state *drbg)
 		drbg->fips_primed = false;
 	}
 }
+EXPORT_SYMBOL(drbg_dealloc_state);
 
 /*
  * Allocate all sub-structures for a DRBG state.
  * The DRBG state structure must already be allocated.
  */
-static inline int drbg_alloc_state(struct drbg_state *drbg)
+int drbg_alloc_state(struct drbg_state *drbg)
 {
 	int ret = -ENOMEM;
 	unsigned int sb_size = 0;
@@ -1321,6 +1324,7 @@ static inline int drbg_alloc_state(struct drbg_state *drbg)
 	drbg_dealloc_state(drbg);
 	return ret;
 }
+EXPORT_SYMBOL(drbg_alloc_state);
 
 /*************************************************************************
  * DRBG interface functions
@@ -1890,8 +1894,7 @@ static int drbg_kcapi_sym_ctr(struct drbg_state *drbg,
  *
  * return: flags
  */
-static inline void drbg_convert_tfm_core(const char *cra_driver_name,
-					 int *coreref, bool *pr)
+void drbg_convert_tfm_core(const char *cra_driver_name, int *coreref, bool *pr)
 {
 	int i = 0;
 	size_t start = 0;
@@ -1918,6 +1921,7 @@ static inline void drbg_convert_tfm_core(const char *cra_driver_name,
 		}
 	}
 }
+EXPORT_SYMBOL(drbg_convert_tfm_core);
 
 static int drbg_kcapi_init(struct crypto_tfm *tfm)
 {
diff --git a/include/crypto/drbg.h b/include/crypto/drbg.h
index c4165126937e..71d53e028e6d 100644
--- a/include/crypto/drbg.h
+++ b/include/crypto/drbg.h
@@ -278,4 +278,11 @@ enum drbg_prefixes {
 	DRBG_PREFIX3
 };
 
+extern int drbg_alloc_state(struct drbg_state *drbg);
+extern void drbg_dealloc_state(struct drbg_state *drbg);
+extern void drbg_convert_tfm_core(const char *cra_driver_name, int *coreref,
+				  bool *pr);
+extern const struct drbg_core drbg_cores[];
+extern unsigned short drbg_sec_strength(drbg_flag_t flags);
+
 #endif /* _DRBG_H */
-- 
2.26.2

[PATCH v36 07/13] LRNG - add SP800-90A DRBG extension

[Stephan Müller]

Using the LRNG switchable DRNG support, the SP800-90A DRBG extension is
implemented.

The DRBG uses the kernel crypto API DRBG implementation. In addition, it
uses the kernel crypto API SHASH support to provide the hashing
operation.

The DRBG supports the choice of either a CTR DRBG using AES-256, HMAC
DRBG with SHA-512 core or Hash DRBG with SHA-512 core. The used core can
be selected with the module parameter lrng_drbg_type. The default is the
CTR DRBG.

When compiling the DRBG extension statically, the DRBG is loaded at
late_initcall stage which implies that with the start of user space, the
user space interfaces of getrandom(2), /dev/random and /dev/urandom
provide random data produced by an SP800-90A DRBG.

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 drivers/char/lrng/Kconfig     |  10 ++
 drivers/char/lrng/Makefile    |   1 +
 drivers/char/lrng/lrng_drbg.c | 197 ++++++++++++++++++++++++++++++++++
 3 files changed, 208 insertions(+)
 create mode 100644 drivers/char/lrng/lrng_drbg.c

diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index daa2057248ac..a3c4cd153f35 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -81,6 +81,16 @@ if LRNG_DRNG_SWITCH
 config LRNG_KCAPI_HASH
 	bool
 
+config LRNG_DRBG
+	tristate "SP800-90A support for the LRNG"
+	depends on CRYPTO
+	select CRYPTO_DRBG_MENU
+	select CRYPTO_SHA512
+	select LRNG_KCAPI_HASH
+	help
+	  Enable the SP800-90A DRBG support for the LRNG. Once the
+	  module is loaded, output from /dev/random, /dev/urandom,
+	  getrandom(2), or get_random_bytes_full is provided by a DRBG.
 endif # LRNG_DRNG_SWITCH
 
 endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 40f8826edeeb..6ebd252db12f 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -12,3 +12,4 @@ obj-$(CONFIG_NUMA)		+= lrng_numa.o
 obj-$(CONFIG_SYSCTL)		+= lrng_proc.o
 obj-$(CONFIG_LRNG_DRNG_SWITCH)	+= lrng_switch.o
 obj-$(CONFIG_LRNG_KCAPI_HASH)	+= lrng_kcapi_hash.o
+obj-$(CONFIG_LRNG_DRBG)		+= lrng_drbg.o
diff --git a/drivers/char/lrng/lrng_drbg.c b/drivers/char/lrng/lrng_drbg.c
new file mode 100644
index 000000000000..c428d41af64d
--- /dev/null
+++ b/drivers/char/lrng/lrng_drbg.c
@@ -0,0 +1,197 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Backend for the LRNG providing the cryptographic primitives using the
+ * kernel crypto API and its DRBG.
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <crypto/drbg.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/lrng.h>
+
+#include "lrng_kcapi_hash.h"
+
+/*
+ * Define a DRBG plus a hash / MAC used to extract data from the entropy pool.
+ * For LRNG_HASH_NAME you can use a hash or a MAC (HMAC or CMAC) of your choice
+ * (Note, you should use the suggested selections below -- using SHA-1 or MD5
+ * is not wise). The idea is that the used cipher primitive can be selected to
+ * be the same as used for the DRBG. I.e. the LRNG only uses one cipher
+ * primitive using the same cipher implementation with the options offered in
+ * the following. This means, if the CTR DRBG is selected and AES-NI is present,
+ * both the CTR DRBG and the selected cmac(aes) use AES-NI.
+ *
+ * The security strengths of the DRBGs are all 256 bits according to
+ * SP800-57 section 5.6.1.
+ *
+ * This definition is allowed to be changed.
+ */
+#ifdef CONFIG_CRYPTO_DRBG_CTR
+static unsigned int lrng_drbg_type = 0;
+#elif defined CONFIG_CRYPTO_DRBG_HMAC
+static unsigned int lrng_drbg_type = 1;
+#elif defined CONFIG_CRYPTO_DRBG_HASH
+static unsigned int lrng_drbg_type = 2;
+#else
+#error "Unknown DRBG in use"
+#endif
+
+/* The parameter must be r/o in sysfs as otherwise races appear. */
+module_param(lrng_drbg_type, uint, 0444);
+MODULE_PARM_DESC(lrng_drbg_type, "DRBG type used for LRNG (0->CTR_DRBG, 1->HMAC_DRBG, 2->Hash_DRBG)");
+
+struct lrng_drbg {
+	const char *hash_name;
+	const char *drbg_core;
+};
+
+static const struct lrng_drbg lrng_drbg_types[] = {
+	{	/* CTR_DRBG with AES-256 using derivation function */
+		.hash_name = "sha512",
+		.drbg_core = "drbg_nopr_ctr_aes256",
+	}, {	/* HMAC_DRBG with SHA-512 */
+		.hash_name = "sha512",
+		.drbg_core = "drbg_nopr_hmac_sha512",
+	}, {	/* Hash_DRBG with SHA-512 using derivation function */
+		.hash_name = "sha512",
+		.drbg_core = "drbg_nopr_sha512"
+	}
+};
+
+static int lrng_drbg_drng_seed_helper(void *drng, const u8 *inbuf, u32 inbuflen)
+{
+	struct drbg_state *drbg = (struct drbg_state *)drng;
+	LIST_HEAD(seedlist);
+	struct drbg_string data;
+	int ret;
+
+	drbg_string_fill(&data, inbuf, inbuflen);
+	list_add_tail(&data.list, &seedlist);
+	ret = drbg->d_ops->update(drbg, &seedlist, drbg->seeded);
+
+	if (ret >= 0)
+		drbg->seeded = true;
+
+	return ret;
+}
+
+static int lrng_drbg_drng_generate_helper(void *drng, u8 *outbuf, u32 outbuflen)
+{
+	struct drbg_state *drbg = (struct drbg_state *)drng;
+
+	return drbg->d_ops->generate(drbg, outbuf, outbuflen, NULL);
+}
+
+static void *lrng_drbg_drng_alloc(u32 sec_strength)
+{
+	struct drbg_state *drbg;
+	int coreref = -1;
+	bool pr = false;
+	int ret;
+
+	drbg_convert_tfm_core(lrng_drbg_types[lrng_drbg_type].drbg_core,
+			      &coreref, &pr);
+	if (coreref < 0)
+		return ERR_PTR(-EFAULT);
+
+	drbg = kzalloc(sizeof(struct drbg_state), GFP_KERNEL);
+	if (!drbg)
+		return ERR_PTR(-ENOMEM);
+
+	drbg->core = &drbg_cores[coreref];
+	drbg->seeded = false;
+	ret = drbg_alloc_state(drbg);
+	if (ret)
+		goto err;
+
+	if (sec_strength > drbg_sec_strength(drbg->core->flags)) {
+		pr_err("Security strength of DRBG (%u bits) lower than requested by LRNG (%u bits)\n",
+			drbg_sec_strength(drbg->core->flags) * 8,
+			sec_strength * 8);
+		goto dealloc;
+	}
+
+	if (sec_strength < drbg_sec_strength(drbg->core->flags))
+		pr_warn("Security strength of DRBG (%u bits) higher than requested by LRNG (%u bits)\n",
+			drbg_sec_strength(drbg->core->flags) * 8,
+			sec_strength * 8);
+
+	pr_info("DRBG with %s core allocated\n", drbg->core->backend_cra_name);
+
+	return drbg;
+
+dealloc:
+	if (drbg->d_ops)
+		drbg->d_ops->crypto_fini(drbg);
+	drbg_dealloc_state(drbg);
+err:
+	kfree(drbg);
+	return ERR_PTR(-EINVAL);
+}
+
+static void lrng_drbg_drng_dealloc(void *drng)
+{
+	struct drbg_state *drbg = (struct drbg_state *)drng;
+
+	if (drbg && drbg->d_ops)
+		drbg->d_ops->crypto_fini(drbg);
+	drbg_dealloc_state(drbg);
+	kfree_sensitive(drbg);
+	pr_info("DRBG deallocated\n");
+}
+
+static void *lrng_drbg_hash_alloc(void)
+{
+	return lrng_kcapi_hash_alloc(lrng_drbg_types[lrng_drbg_type].hash_name);
+}
+
+static const char *lrng_drbg_name(void)
+{
+	return lrng_drbg_types[lrng_drbg_type].drbg_core;
+}
+
+static const char *lrng_hash_name(void)
+{
+	return lrng_drbg_types[lrng_drbg_type].hash_name;
+}
+
+static const struct lrng_crypto_cb lrng_drbg_crypto_cb = {
+	.lrng_drng_name			= lrng_drbg_name,
+	.lrng_hash_name			= lrng_hash_name,
+	.lrng_drng_alloc		= lrng_drbg_drng_alloc,
+	.lrng_drng_dealloc		= lrng_drbg_drng_dealloc,
+	.lrng_drng_seed_helper		= lrng_drbg_drng_seed_helper,
+	.lrng_drng_generate_helper	= lrng_drbg_drng_generate_helper,
+	.lrng_hash_alloc		= lrng_drbg_hash_alloc,
+	.lrng_hash_dealloc		= lrng_kcapi_hash_dealloc,
+	.lrng_hash_digestsize		= lrng_kcapi_hash_digestsize,
+	.lrng_hash_init			= lrng_kcapi_hash_init,
+	.lrng_hash_update		= lrng_kcapi_hash_update,
+	.lrng_hash_final		= lrng_kcapi_hash_final,
+};
+
+static int __init lrng_drbg_init(void)
+{
+	if (lrng_drbg_type >= ARRAY_SIZE(lrng_drbg_types)) {
+		pr_err("lrng_drbg_type parameter too large (given %u - max: %lu)",
+		       lrng_drbg_type,
+		       (unsigned long)ARRAY_SIZE(lrng_drbg_types) - 1);
+		return -EAGAIN;
+	}
+	return lrng_set_drng_cb(&lrng_drbg_crypto_cb);
+}
+
+static void __exit lrng_drbg_exit(void)
+{
+	lrng_set_drng_cb(NULL);
+}
+
+late_initcall(lrng_drbg_init);
+module_exit(lrng_drbg_exit);
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_AUTHOR("Stephan Mueller <smueller@chronox.de>");
+MODULE_DESCRIPTION("Linux Random Number Generator - SP800-90A DRBG backend");
-- 
2.26.2

[PATCH v36 08/13] LRNG - add kernel crypto API PRNG extension

[Stephan Müller]

Add runtime-pluggable support for all PRNGs that are accessible via
the kernel crypto API, including hardware PRNGs. The PRNG is selected
with the module parameter drng_name where the name must be one that the
kernel crypto API can resolve into an RNG.

This allows using of the kernel crypto API PRNG implementations that
provide an interface to hardware PRNGs. Using this extension,
the LRNG uses the hardware PRNGs to generate random numbers. An
example is the S390 CPACF support providing such a PRNG.

The hash is provided by a kernel crypto API SHASH whose digest size
complies with the seedsize of the PRNG.

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 drivers/char/lrng/Kconfig      |  13 ++
 drivers/char/lrng/Makefile     |   1 +
 drivers/char/lrng/lrng_kcapi.c | 228 +++++++++++++++++++++++++++++++++
 3 files changed, 242 insertions(+)
 create mode 100644 drivers/char/lrng/lrng_kcapi.c

diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index a3c4cd153f35..af487b3391e0 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -91,6 +91,19 @@ config LRNG_DRBG
 	  Enable the SP800-90A DRBG support for the LRNG. Once the
 	  module is loaded, output from /dev/random, /dev/urandom,
 	  getrandom(2), or get_random_bytes_full is provided by a DRBG.
+
+config LRNG_KCAPI
+	tristate "Kernel Crypto API support for the LRNG"
+	depends on CRYPTO
+	depends on !LRNG_DRBG
+	select CRYPTO_RNG
+	select LRNG_KCAPI_HASH
+	help
+	  Enable the support for generic pseudo-random number
+	  generators offered by the kernel crypto API with the
+	  LRNG. Once the module is loaded, output from /dev/random,
+	  /dev/urandom, getrandom(2), or get_random_bytes is
+	  provided by the selected kernel crypto API RNG.
 endif # LRNG_DRNG_SWITCH
 
 endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 6ebd252db12f..97d2b13d3227 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -13,3 +13,4 @@ obj-$(CONFIG_SYSCTL)		+= lrng_proc.o
 obj-$(CONFIG_LRNG_DRNG_SWITCH)	+= lrng_switch.o
 obj-$(CONFIG_LRNG_KCAPI_HASH)	+= lrng_kcapi_hash.o
 obj-$(CONFIG_LRNG_DRBG)		+= lrng_drbg.o
+obj-$(CONFIG_LRNG_KCAPI)	+= lrng_kcapi.o
diff --git a/drivers/char/lrng/lrng_kcapi.c b/drivers/char/lrng/lrng_kcapi.c
new file mode 100644
index 000000000000..197dc84552e9
--- /dev/null
+++ b/drivers/char/lrng/lrng_kcapi.c
@@ -0,0 +1,228 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Backend for the LRNG providing the cryptographic primitives using the
+ * kernel crypto API.
+ *
+ * Copyright (C) 2018 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <crypto/hash.h>
+#include <crypto/rng.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/lrng.h>
+
+#include "lrng_kcapi_hash.h"
+
+static char *drng_name = NULL;
+module_param(drng_name, charp, 0444);
+MODULE_PARM_DESC(drng_name, "Kernel crypto API name of DRNG");
+
+static char *pool_hash = "sha512";
+module_param(pool_hash, charp, 0444);
+MODULE_PARM_DESC(pool_hash,
+		 "Kernel crypto API name of hash or keyed message digest to read the entropy pool");
+
+static char *seed_hash = NULL;
+module_param(seed_hash, charp, 0444);
+MODULE_PARM_DESC(seed_hash,
+		 "Kernel crypto API name of hash with output size equal to seedsize of DRNG to bring seed string to the size required by the DRNG");
+
+struct lrng_drng_info {
+	struct crypto_rng *kcapi_rng;
+	void *lrng_hash;
+};
+
+static void *lrng_kcapi_drng_hash_alloc(void)
+{
+	return lrng_kcapi_hash_alloc(pool_hash);
+}
+
+static int lrng_kcapi_drng_seed_helper(void *drng, const u8 *inbuf,
+				       u32 inbuflen)
+{
+	SHASH_DESC_ON_STACK(shash, NULL);
+	struct lrng_drng_info *lrng_drng_info = (struct lrng_drng_info *)drng;
+	struct crypto_rng *kcapi_rng = lrng_drng_info->kcapi_rng;
+	void *hash = lrng_drng_info->lrng_hash;
+	u32 digestsize = lrng_kcapi_hash_digestsize(hash);
+	u8 digest[64] __aligned(8);
+	int ret;
+
+	if (!hash)
+		return crypto_rng_reset(kcapi_rng, inbuf, inbuflen);
+
+	BUG_ON(digestsize > sizeof(digest));
+
+	ret = lrng_kcapi_hash_init(shash, hash) ?:
+	      lrng_kcapi_hash_update(shash, inbuf, inbuflen) ?:
+	      lrng_kcapi_hash_final(shash, digest);
+	if (ret)
+		return ret;
+
+	ret = crypto_rng_reset(kcapi_rng, digest, digestsize);
+	if (ret)
+		return ret;
+
+	memzero_explicit(digest, digestsize);
+	return 0;
+}
+
+static int lrng_kcapi_drng_generate_helper(void *drng, u8 *outbuf,
+					   u32 outbuflen)
+{
+	struct lrng_drng_info *lrng_drng_info = (struct lrng_drng_info *)drng;
+	struct crypto_rng *kcapi_rng = lrng_drng_info->kcapi_rng;
+	int ret = crypto_rng_get_bytes(kcapi_rng, outbuf, outbuflen);
+
+	if (ret < 0)
+		return ret;
+
+	return outbuflen;
+}
+
+static void *lrng_kcapi_drng_alloc(u32 sec_strength)
+{
+	struct lrng_drng_info *lrng_drng_info;
+	struct crypto_rng *kcapi_rng;
+	int seedsize;
+	void *ret =  ERR_PTR(-ENOMEM);
+
+	if (!drng_name) {
+		pr_err("DRNG name missing\n");
+		return ERR_PTR(-EINVAL);
+	}
+
+	if (!memcmp(drng_name, "drbg", 4)) {
+		pr_err("SP800-90A DRBG cannot be allocated using lrng_kcapi backend, use lrng_drbg backend instead\n");
+		return ERR_PTR(-EINVAL);
+	}
+
+	if (!memcmp(drng_name, "stdrng", 6)) {
+		pr_err("stdrng cannot be allocated using lrng_kcapi backend, it is too unspecific and potentially may allocate the DRBG\n");
+		return ERR_PTR(-EINVAL);
+	}
+
+	lrng_drng_info = kmalloc(sizeof(*lrng_drng_info), GFP_KERNEL);
+	if (!lrng_drng_info)
+		return ERR_PTR(-ENOMEM);
+
+	kcapi_rng = crypto_alloc_rng(drng_name, 0, 0);
+	if (IS_ERR(kcapi_rng)) {
+		pr_err("DRNG %s cannot be allocated\n", drng_name);
+		ret = ERR_CAST(kcapi_rng);
+		goto free;
+	}
+	lrng_drng_info->kcapi_rng = kcapi_rng;
+
+	seedsize =  crypto_rng_seedsize(kcapi_rng);
+
+	if (sec_strength > seedsize)
+		pr_info("Seedsize DRNG (%u bits) lower than security strength of LRNG noise source (%u bits)\n",
+			crypto_rng_seedsize(kcapi_rng) * 8, sec_strength * 8);
+
+	if (seedsize) {
+		void *lrng_hash;
+
+		if (!seed_hash) {
+			switch (seedsize) {
+			case 32:
+				seed_hash = "sha256";
+				break;
+			case 48:
+				seed_hash = "sha384";
+				break;
+			case 64:
+				seed_hash = "sha512";
+				break;
+			default:
+				pr_err("Seed size %d cannot be processed\n",
+				       seedsize);
+				goto dealloc;
+			}
+		}
+
+		lrng_hash = lrng_kcapi_hash_alloc(seed_hash);
+		if (IS_ERR(lrng_hash)) {
+			ret = ERR_CAST(lrng_hash);
+			goto dealloc;
+		}
+
+		if (seedsize != lrng_kcapi_hash_digestsize(lrng_hash)) {
+			pr_err("Seed hash output size not equal to DRNG seed size\n");
+			lrng_kcapi_hash_dealloc(lrng_hash);
+			ret = ERR_PTR(-EINVAL);
+			goto dealloc;
+		}
+
+		lrng_drng_info->lrng_hash = lrng_hash;
+
+		pr_info("Seed hash %s allocated\n", seed_hash);
+	} else {
+		lrng_drng_info->lrng_hash = NULL;
+	}
+
+	pr_info("Kernel crypto API DRNG %s allocated\n", drng_name);
+
+	return lrng_drng_info;
+
+dealloc:
+	crypto_free_rng(kcapi_rng);
+free:
+	kfree(lrng_drng_info);
+	return ret;
+}
+
+static void lrng_kcapi_drng_dealloc(void *drng)
+{
+	struct lrng_drng_info *lrng_drng_info = (struct lrng_drng_info *)drng;
+	struct crypto_rng *kcapi_rng = lrng_drng_info->kcapi_rng;
+
+	crypto_free_rng(kcapi_rng);
+	if (lrng_drng_info->lrng_hash)
+		lrng_kcapi_hash_dealloc(lrng_drng_info->lrng_hash);
+	kfree(lrng_drng_info);
+	pr_info("DRNG %s deallocated\n", drng_name);
+}
+
+static const char *lrng_kcapi_drng_name(void)
+{
+	return drng_name;
+}
+
+static const char *lrng_kcapi_pool_hash(void)
+{
+	return pool_hash;
+}
+
+static const struct lrng_crypto_cb lrng_kcapi_crypto_cb = {
+	.lrng_drng_name			= lrng_kcapi_drng_name,
+	.lrng_hash_name			= lrng_kcapi_pool_hash,
+	.lrng_drng_alloc		= lrng_kcapi_drng_alloc,
+	.lrng_drng_dealloc		= lrng_kcapi_drng_dealloc,
+	.lrng_drng_seed_helper		= lrng_kcapi_drng_seed_helper,
+	.lrng_drng_generate_helper	= lrng_kcapi_drng_generate_helper,
+	.lrng_hash_alloc		= lrng_kcapi_drng_hash_alloc,
+	.lrng_hash_dealloc		= lrng_kcapi_hash_dealloc,
+	.lrng_hash_digestsize		= lrng_kcapi_hash_digestsize,
+	.lrng_hash_init			= lrng_kcapi_hash_init,
+	.lrng_hash_update		= lrng_kcapi_hash_update,
+	.lrng_hash_final		= lrng_kcapi_hash_final,
+};
+
+static int __init lrng_kcapi_init(void)
+{
+	return lrng_set_drng_cb(&lrng_kcapi_crypto_cb);
+}
+static void __exit lrng_kcapi_exit(void)
+{
+	lrng_set_drng_cb(NULL);
+}
+
+late_initcall(lrng_kcapi_init);
+module_exit(lrng_kcapi_exit);
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_AUTHOR("Stephan Mueller <smueller@chronox.de>");
+MODULE_DESCRIPTION("Linux Random Number Generator - kernel crypto API DRNG backend");
-- 
2.26.2

[PATCH v36 09/13] crypto: provide access to a static Jitter RNG state

[Stephan Müller]

To support the LRNG operation which uses the Jitter RNG separately
from the kernel crypto API, at a time where potentially the regular
memory management is not yet initialized, the Jitter RNG needs to
provide a state whose memory is defined at compile time. As only once
instance will ever be needed by the LRNG, define once static memory
block which is solely to be used by the LRNG.

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 crypto/jitterentropy-kcapi.c                  |  3 +-
 crypto/jitterentropy.c                        | 31 ++++++++++++++++++-
 .../crypto/internal}/jitterentropy.h          |  3 ++
 3 files changed, 34 insertions(+), 3 deletions(-)
 rename {crypto => include/crypto/internal}/jitterentropy.h (84%)

diff --git a/crypto/jitterentropy-kcapi.c b/crypto/jitterentropy-kcapi.c
index eb7d1dd506bf..25a192f5984e 100644
--- a/crypto/jitterentropy-kcapi.c
+++ b/crypto/jitterentropy-kcapi.c
@@ -43,8 +43,7 @@
 #include <linux/time.h>
 #include <linux/crypto.h>
 #include <crypto/internal/rng.h>
-
-#include "jitterentropy.h"
+#include <crypto/internal/jitterentropy.h>
 
 /***************************************************************************
  * Helper function
diff --git a/crypto/jitterentropy.c b/crypto/jitterentropy.c
index 6e147c43fc18..fa1459f09b01 100644
--- a/crypto/jitterentropy.c
+++ b/crypto/jitterentropy.c
@@ -117,7 +117,7 @@ struct rand_data {
 #define JENT_EHEALTH		9 /* Health test failed during initialization */
 #define JENT_ERCT		10 /* RCT failed during initialization */
 
-#include "jitterentropy.h"
+#include <crypto/internal/jitterentropy.h>
 
 /***************************************************************************
  * Adaptive Proportion Test
@@ -854,3 +854,32 @@ int jent_entropy_init(void)
 
 	return 0;
 }
+
+struct rand_data *jent_lrng_entropy_collector(void)
+{
+	static unsigned char lrng_jent_mem[JENT_MEMORY_SIZE];
+	static struct rand_data lrng_jent_state = {
+		.data		= 0,
+		.old_data	= 0,
+		.prev_time	= 0,
+		.last_delta	= 0,
+		.last_delta2	= 0,
+		.osr		= 1,
+		.mem		= lrng_jent_mem,
+		.memlocation	= 0,
+		.memblocks	= JENT_MEMORY_BLOCKSIZE,
+		.memblocksize	= JENT_MEMORY_BLOCKS,
+		.memaccessloops	= JENT_MEMORY_ACCESSLOOPS,
+		.rct_count	= 0,
+		.apt_observations = 0,
+		.apt_count	= 0,
+		.apt_base	= 0,
+		.apt_base_set	= 0,
+		.health_failure = 0
+	};
+
+	if (jent_entropy_init())
+		return NULL;
+
+	return &lrng_jent_state;
+}
diff --git a/crypto/jitterentropy.h b/include/crypto/internal/jitterentropy.h
similarity index 84%
rename from crypto/jitterentropy.h
rename to include/crypto/internal/jitterentropy.h
index c83fff32d130..6e07d86eac82 100644
--- a/crypto/jitterentropy.h
+++ b/include/crypto/internal/jitterentropy.h
@@ -15,3 +15,6 @@ extern int jent_read_entropy(struct rand_data *ec, unsigned char *data,
 extern struct rand_data *jent_entropy_collector_alloc(unsigned int osr,
 						      unsigned int flags);
 extern void jent_entropy_collector_free(struct rand_data *entropy_collector);
+
+/* Access to statically allocated Jitter RNG instance */
+extern struct rand_data *jent_lrng_entropy_collector(void);
-- 
2.26.2

[PATCH v36 10/13] LRNG - add Jitter RNG fast noise source

[Stephan Müller]

The Jitter RNG fast noise source implemented as part of the kernel
crypto API is queried for 256 bits of entropy at the time the seed
buffer managed by the LRNG is about to be filled.

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 drivers/char/lrng/Kconfig     | 12 +++++
 drivers/char/lrng/Makefile    |  1 +
 drivers/char/lrng/lrng_jent.c | 88 +++++++++++++++++++++++++++++++++++
 3 files changed, 101 insertions(+)
 create mode 100644 drivers/char/lrng/lrng_jent.c

diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index af487b3391e0..87682f57efb8 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -106,4 +106,16 @@ config LRNG_KCAPI
 	  provided by the selected kernel crypto API RNG.
 endif # LRNG_DRNG_SWITCH
 
+config LRNG_JENT
+	bool "Enable Jitter RNG as LRNG Seed Source"
+	depends on CRYPTO
+	select CRYPTO_JITTERENTROPY
+	help
+	  The Linux RNG may use the Jitter RNG as noise source. Enabling
+	  this option enables the use of the Jitter RNG. Its default
+	  entropy level is 16 bits of entropy per 256 data bits delivered
+	  by the Jitter RNG. This entropy level can be changed at boot
+	  time or at runtime with the lrng_base.jitterrng configuration
+	  variable.
+
 endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 97d2b13d3227..6be88156010a 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -14,3 +14,4 @@ obj-$(CONFIG_LRNG_DRNG_SWITCH)	+= lrng_switch.o
 obj-$(CONFIG_LRNG_KCAPI_HASH)	+= lrng_kcapi_hash.o
 obj-$(CONFIG_LRNG_DRBG)		+= lrng_drbg.o
 obj-$(CONFIG_LRNG_KCAPI)	+= lrng_kcapi.o
+obj-$(CONFIG_LRNG_JENT)		+= lrng_jent.o
diff --git a/drivers/char/lrng/lrng_jent.c b/drivers/char/lrng/lrng_jent.c
new file mode 100644
index 000000000000..225505271fcb
--- /dev/null
+++ b/drivers/char/lrng/lrng_jent.c
@@ -0,0 +1,88 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG Fast Noise Source: Jitter RNG
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/types.h>
+#include <crypto/internal/jitterentropy.h>
+
+#include "lrng_internal.h"
+
+/*
+ * Estimated entropy of data is a 16th of LRNG_DRNG_SECURITY_STRENGTH_BITS.
+ * Albeit a full entropy assessment is provided for the noise source indicating
+ * that it provides high entropy rates and considering that it deactivates
+ * when it detects insufficient hardware, the chosen under estimation of
+ * entropy is considered to be acceptable to all reviewers.
+ */
+static u32 jitterrng = LRNG_DRNG_SECURITY_STRENGTH_BITS>>4;
+module_param(jitterrng, uint, 0644);
+MODULE_PARM_DESC(jitterrng, "Entropy in bits of 256 data bits from Jitter RNG noise source");
+
+/**
+ * lrng_get_jent() - Get Jitter RNG entropy
+ *
+ * @outbuf: buffer to store entropy
+ * @outbuflen: length of buffer
+ *
+ * Return:
+ * * > 0 on success where value provides the added entropy in bits
+ * * 0 if no fast source was available
+ */
+static struct rand_data *lrng_jent_state;
+
+u32 lrng_get_jent(u8 *outbuf, unsigned int outbuflen)
+{
+	int ret;
+	u32 ent_bits = jitterrng;
+	unsigned long flags;
+	static DEFINE_SPINLOCK(lrng_jent_lock);
+	static int lrng_jent_initialized = 0;
+
+	spin_lock_irqsave(&lrng_jent_lock, flags);
+
+	if (!ent_bits || (lrng_jent_initialized == -1)) {
+		spin_unlock_irqrestore(&lrng_jent_lock, flags);
+		return 0;
+	}
+
+	if (!lrng_jent_initialized) {
+		lrng_jent_state = jent_lrng_entropy_collector();
+		if (!lrng_jent_state) {
+			jitterrng = 0;
+			lrng_jent_initialized = -1;
+			spin_unlock_irqrestore(&lrng_jent_lock, flags);
+			pr_info("Jitter RNG unusable on current system\n");
+			return 0;
+		}
+		lrng_jent_initialized = 1;
+		pr_debug("Jitter RNG working on current system\n");
+	}
+	ret = jent_read_entropy(lrng_jent_state, outbuf, outbuflen);
+	spin_unlock_irqrestore(&lrng_jent_lock, flags);
+
+	if (ret) {
+		pr_debug("Jitter RNG failed with %d\n", ret);
+		return 0;
+	}
+
+	/* Obtain entropy statement */
+	if (outbuflen != LRNG_DRNG_SECURITY_STRENGTH_BYTES)
+		ent_bits = (ent_bits * outbuflen<<3) /
+			   LRNG_DRNG_SECURITY_STRENGTH_BITS;
+	/* Cap entropy to buffer size in bits */
+	ent_bits = min_t(u32, ent_bits, outbuflen<<3);
+	pr_debug("obtained %u bits of entropy from Jitter RNG noise source\n",
+		 ent_bits);
+
+	return ent_bits;
+}
+
+u32 lrng_jent_entropylevel(void)
+{
+	return min_t(u32, jitterrng, LRNG_DRNG_SECURITY_STRENGTH_BITS);
+}
-- 
2.26.2

[PATCH v36 11/13] LRNG - add SP800-90B compliant health tests

[Stephan Müller]

Implement health tests for LRNG's slow noise sources as mandated by
SP-800-90B The file contains the following health tests:

- stuck test: The stuck test calculates the first, second and third
  discrete derivative of the time stamp to be processed by the hash
  for the per-CPU entropy pool. Only if all three values are non-zero,
  the received time delta is considered to be non-stuck.

- SP800-90B Repetition Count Test (RCT): The LRNG uses an enhanced
  version of the RCT specified in SP800-90B section 4.4.1. Instead of
  counting identical back-to-back values, the input to the RCT is the
  counting of the stuck values during the processing of received
  interrupt events. The RCT is applied with alpha=2^-30 compliant to
  the recommendation of FIPS 140-2 IG 9.8. During the counting operation,
  the LRNG always calculates the RCT cut-off value of C. If that value
  exceeds the allowed cut-off value, the LRNG will trigger the health
  test failure discussed below. An error is logged to the kernel log
  that such RCT failure occurred. This test is only applied and
  enforced in FIPS mode, i.e. when the kernel compiled with
  CONFIG_CONFIG_FIPS is started with fips=1.

- SP800-90B Adaptive Proportion Test (APT): The LRNG implements the
  APT as defined in SP800-90B section 4.4.2. The applied significance
  level again is alpha=2^-30 compliant to the recommendation of FIPS
  140-2 IG 9.8.

The aforementioned health tests are applied to the first 1,024 time stamps
obtained from interrupt events. In case one error is identified for either
the RCT, or the APT, the collected entropy is invalidated and the
SP800-90B startup health test is restarted.

As long as the SP800-90B startup health test is not completed, all LRNG
random number output interfaces that may block will block and not generate
any data. This implies that only those potentially blocking interfaces are
defined to provide random numbers that are seeded with the interrupt noise
source being SP800-90B compliant. All other output interfaces will not be
affected by the SP800-90B startup test and thus are not considered
SP800-90B compliant.

At runtime, the SP800-90B APT and RCT are applied to each time stamp
generated for a received interrupt. When either the APT and RCT indicates
a noise source failure, the LRNG is reset to a state it has immediately
after boot:

- all entropy counters are set to zero

- the SP800-90B startup tests are re-performed which implies that
getrandom(2) would block again until new entropy was collected

To summarize, the following rules apply:

• SP800-90B compliant output interfaces

  - /dev/random

  - getrandom(2) system call

  -  get_random_bytes kernel-internal interface when being triggered by
     the callback registered with add_random_ready_callback

• SP800-90B non-compliant output interfaces

  - /dev/urandom

  - get_random_bytes kernel-internal interface called directly

  - randomize_page kernel-internal interface

  - get_random_u32 and get_random_u64 kernel-internal interfaces

  - get_random_u32_wait, get_random_u64_wait, get_random_int_wait, and
    get_random_long_wait kernel-internal interfaces

If either the RCT, or the APT health test fails irrespective whether
during initialization or runtime, the following actions occur:

  1. The entropy of the entire entropy pool is invalidated.

  2. All DRNGs are reset which imply that they are treated as being
     not seeded and require a reseed during next invocation.

  3. The SP800-90B startup health test are initiated with all
     implications of the startup tests. That implies that from that point
     on, new events must be observed and its entropy must be inserted into
     the entropy pool before random numbers are calculated from the
     entropy pool.

Further details on the SP800-90B compliance and the availability of all
test tools required to perform all tests mandated by SP800-90B are
provided at [1].

The entire health testing code is compile-time configurable.

The patch provides a CONFIG_BROKEN configuration of the APT / RCT cutoff
values which have a high likelihood to trigger the health test failure.
The BROKEN APT cutoff is set to the exact mean of the expected value if
the time stamps are equally distributed (512 time stamps divided by 16
possible values due to using the 4 LSB of the time stamp). The BROKEN
RCT cutoff value is set to 1 which is likely to be triggered during
regular operation.

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 drivers/char/lrng/Kconfig       |  56 +++++
 drivers/char/lrng/Makefile      |   1 +
 drivers/char/lrng/lrng_health.c | 407 ++++++++++++++++++++++++++++++++
 3 files changed, 464 insertions(+)
 create mode 100644 drivers/char/lrng/lrng_health.c

diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index 87682f57efb8..a059da0b2f5d 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -118,4 +118,60 @@ config LRNG_JENT
 	  time or at runtime with the lrng_base.jitterrng configuration
 	  variable.
 
+config LRNG_HEALTH_TESTS
+	bool "Enable noise source online health tests"
+	help
+	  The online health tests validate the noise source at
+	  runtime for fatal errors. These tests include SP800-90B
+	  compliant tests which are invoked if the system is booted
+	  with fips=1. In case of fatal errors during active
+	  SP800-90B tests, the issue is logged and the noise
+	  data is discarded. These tests are required for full
+	  compliance with SP800-90B.
+
+	  If unsure, say Y.
+
+config LRNG_RCT_BROKEN
+	bool "SP800-90B RCT with dangerous low cutoff value"
+	depends on LRNG_HEALTH_TESTS
+	depends on BROKEN
+	default n
+	help
+	  This option enables a dangerously low SP800-90B repetitive
+	  count test (RCT) cutoff value which makes it very likely
+	  that the RCT is triggered to raise a self test failure.
+
+	  This option is ONLY intended for developers wanting to
+	  test the effectiveness of the SP800-90B RCT health test.
+
+	  If unsure, say N.
+
+config LRNG_APT_BROKEN
+	bool "SP800-90B APT with dangerous low cutoff value"
+	depends on LRNG_HEALTH_TESTS
+	depends on BROKEN
+	default n
+	help
+	  This option enables a dangerously low SP800-90B adaptive
+	  proportion test (APT) cutoff value which makes it very
+	  likely that the APT is triggered to raise a self test
+	  failure.
+
+	  This option is ONLY intended for developers wanting to
+	  test the effectiveness of the SP800-90B APT health test.
+
+	  If unsure, say N.
+
+# Default taken from SP800-90B sec 4.4.1 - significance level 2^-30
+config LRNG_RCT_CUTOFF
+	int
+	default 31 if !LRNG_RCT_BROKEN
+	default 1 if LRNG_RCT_BROKEN
+
+# Default taken from SP800-90B sec 4.4.2 - significance level 2^-30
+config LRNG_APT_CUTOFF
+	int
+	default 325 if !LRNG_APT_BROKEN
+	default 32 if LRNG_APT_BROKEN
+
 endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 6be88156010a..3c8637befb42 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -15,3 +15,4 @@ obj-$(CONFIG_LRNG_KCAPI_HASH)	+= lrng_kcapi_hash.o
 obj-$(CONFIG_LRNG_DRBG)		+= lrng_drbg.o
 obj-$(CONFIG_LRNG_KCAPI)	+= lrng_kcapi.o
 obj-$(CONFIG_LRNG_JENT)		+= lrng_jent.o
+obj-$(CONFIG_LRNG_HEALTH_TESTS)	+= lrng_health.o
diff --git a/drivers/char/lrng/lrng_health.c b/drivers/char/lrng/lrng_health.c
new file mode 100644
index 000000000000..7817aa6f3357
--- /dev/null
+++ b/drivers/char/lrng/lrng_health.c
@@ -0,0 +1,407 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Linux Random Number Generator (LRNG) Health Testing
+ *
+ * Copyright (C) 2019 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/fips.h>
+#include <linux/module.h>
+
+#include "lrng_internal.h"
+
+/* Stuck Test */
+struct lrng_stuck_test {
+	u32 last_time;		/* Stuck test: time of previous IRQ */
+	u32 last_delta;		/* Stuck test: delta of previous IRQ */
+	u32 last_delta2;	/* Stuck test: 2. time derivation of prev IRQ */
+};
+
+/* Repetition Count Test */
+struct lrng_rct {
+	atomic_t rct_count;	/* Number of stuck values */
+};
+
+/* Adaptive Proportion Test */
+struct lrng_apt {
+	/* Data window size */
+#define LRNG_APT_WINDOW_SIZE	512
+	/* LSB of time stamp to process */
+#define LRNG_APT_LSB		16
+#define LRNG_APT_WORD_MASK	(LRNG_APT_LSB - 1)
+	atomic_t apt_count;		/* APT counter */
+	atomic_t apt_base;		/* APT base reference */
+
+	atomic_t apt_trigger;
+	bool apt_base_set;	/* Is APT base set? */
+};
+
+/* The health test code must operate lock-less */
+struct lrng_health {
+	struct lrng_rct rct;
+	struct lrng_apt apt;
+
+	bool health_test_enabled;
+
+	/* SP800-90B startup health tests */
+#define LRNG_SP80090B_STARTUP_SAMPLES  1024
+#define LRNG_SP80090B_STARTUP_BLOCKS   ((LRNG_SP80090B_STARTUP_SAMPLES + \
+					 LRNG_APT_WINDOW_SIZE - 1) /    \
+					LRNG_APT_WINDOW_SIZE)
+	bool sp80090b_startup_done;
+	atomic_t sp80090b_startup_blocks;
+};
+
+static struct lrng_health lrng_health = {
+	.rct.rct_count = ATOMIC_INIT(0),
+
+	.apt.apt_count = ATOMIC_INIT(0),
+	.apt.apt_base = ATOMIC_INIT(-1),
+	.apt.apt_trigger = ATOMIC_INIT(LRNG_APT_WINDOW_SIZE),
+	.apt.apt_base_set = false,
+
+	.health_test_enabled = true,
+
+	.sp80090b_startup_blocks = ATOMIC_INIT(LRNG_SP80090B_STARTUP_BLOCKS),
+	.sp80090b_startup_done = false,
+};
+
+static DEFINE_PER_CPU(struct lrng_stuck_test, lrng_stuck_test);
+
+static inline bool lrng_sp80090b_health_requested(void)
+{
+	/* Health tests are only requested in FIPS mode */
+	return fips_enabled;
+}
+
+static inline bool lrng_sp80090b_health_enabled(void)
+{
+	struct lrng_health *health = &lrng_health;
+
+	return lrng_sp80090b_health_requested() && health->health_test_enabled;
+}
+
+/***************************************************************************
+ * SP800-90B Compliance
+ *
+ * If the Linux-RNG is booted into FIPS mode, the following interfaces
+ * provide an SP800-90B compliant noise source:
+ *
+ * * /dev/random
+ * * getrandom(2)
+ * * get_random_bytes when using it in conjunction with
+ *   add_random_ready_callback
+ *
+ * All other interfaces, including /dev/urandom or get_random_bytes without
+ * the add_random_ready_callback cannot claim to use an SP800-90B compliant
+ * noise source.
+ ***************************************************************************/
+
+/*
+ * Perform SP800-90B startup testing
+ */
+static inline void lrng_sp80090b_startup(struct lrng_health *health)
+{
+	if (!health->sp80090b_startup_done &&
+	    atomic_dec_and_test(&health->sp80090b_startup_blocks)) {
+		health->sp80090b_startup_done = true;
+		pr_info("SP800-90B startup health tests completed\n");
+		lrng_init_ops(0);
+
+		/*
+		 * Force a reseed of DRNGs to ensure they are seeded with
+		 * entropy that passed the SP800-90B health tests.
+		 * As the DRNG always will reseed before generating
+		 * random numbers, it does not need a reseed trigger.
+		 */
+		lrng_drng_force_reseed();
+	}
+}
+
+/*
+ * Handle failure of SP800-90B startup testing
+ */
+static inline void lrng_sp80090b_startup_failure(struct lrng_health *health)
+{
+	/* Reset of LRNG and its entropy - NOTE: we are in atomic context */
+	lrng_reset();
+
+	/*
+	 * Reset the SP800-90B startup test.
+	 *
+	 * NOTE SP800-90B section 4.3 bullet 4 does not specify what
+	 * exactly is to be done in case of failure! Thus, we do what
+	 * makes sense, i.e. restarting the health test and thus gating
+	 * the output function of /dev/random and getrandom(2).
+	 */
+	atomic_set(&health->sp80090b_startup_blocks,
+		   LRNG_SP80090B_STARTUP_BLOCKS);
+}
+
+/*
+ * Handle failure of SP800-90B runtime testing
+ */
+static inline void lrng_sp80090b_runtime_failure(struct lrng_health *health)
+{
+	lrng_sp80090b_startup_failure(health);
+	health->sp80090b_startup_done = false;
+}
+
+static inline void lrng_sp80090b_failure(struct lrng_health *health)
+{
+	if (health->sp80090b_startup_done) {
+		pr_err("SP800-90B runtime health test failure - invalidating all existing entropy and initiate SP800-90B startup\n");
+		lrng_sp80090b_runtime_failure(health);
+	} else {
+		pr_err("SP800-90B startup test failure - resetting\n");
+		lrng_sp80090b_startup_failure(health);
+	}
+}
+
+/*
+ * Is the SP800-90B startup testing complete?
+ *
+ * This function is called by the LRNG to determine whether to unblock
+ * a certain user interface. Therefore, only the potentially blocking
+ * user interfaces are considered SP800-90B compliant.
+ */
+bool lrng_sp80090b_startup_complete(void)
+{
+	struct lrng_health *health = &lrng_health;
+
+	return (lrng_sp80090b_health_enabled()) ? health->sp80090b_startup_done:
+						  true;
+}
+
+bool lrng_sp80090b_compliant(void)
+{
+	struct lrng_health *health = &lrng_health;
+
+	return lrng_sp80090b_health_enabled() && health->sp80090b_startup_done;
+}
+
+/***************************************************************************
+ * Adaptive Proportion Test
+ *
+ * This test complies with SP800-90B section 4.4.2.
+ ***************************************************************************/
+
+/*
+ * Reset the APT counter
+ *
+ * @health [in] Reference to health state
+ */
+static inline void lrng_apt_reset(struct lrng_health *health,
+				  unsigned int time_masked)
+{
+	struct lrng_apt *apt = &health->apt;
+
+	pr_debug("APT value %d for base %d\n",
+		 atomic_read(&apt->apt_count), atomic_read(&apt->apt_base));
+
+	/* Reset APT */
+	atomic_set(&apt->apt_count, 0);
+	atomic_set(&apt->apt_base, time_masked);
+}
+
+static inline void lrng_apt_restart(struct lrng_health *health)
+{
+	struct lrng_apt *apt = &health->apt;
+
+	atomic_set(&apt->apt_trigger, LRNG_APT_WINDOW_SIZE);
+}
+
+/*
+ * Insert a new entropy event into APT
+ *
+ * This function does is void as it does not decide about the fate of a time
+ * stamp. An APT failure can only happen at the same time of a stuck test
+ * failure. Thus, the stuck failure will already decide how the time stamp
+ * is handled.
+ *
+ * @health [in] Reference to health state
+ * @now_time [in] Time stamp to process
+ */
+static inline void lrng_apt_insert(struct lrng_health *health,
+				   unsigned int now_time)
+{
+	struct lrng_apt *apt = &health->apt;
+
+	if (!lrng_sp80090b_health_requested())
+		return;
+
+	now_time &= LRNG_APT_WORD_MASK;
+
+	/* Initialization of APT */
+	if (!apt->apt_base_set) {
+		atomic_set(&apt->apt_base, now_time);
+		apt->apt_base_set = true;
+		return;
+	}
+
+	if (now_time == (unsigned int)atomic_read(&apt->apt_base)) {
+		u32 apt_val = (u32)atomic_inc_return_relaxed(&apt->apt_count);
+
+		if (apt_val >= CONFIG_LRNG_APT_CUTOFF)
+			lrng_sp80090b_failure(health);
+	}
+
+	if (atomic_dec_and_test(&apt->apt_trigger)) {
+		lrng_apt_restart(health);
+		lrng_apt_reset(health, now_time);
+		lrng_sp80090b_startup(health);
+	}
+}
+
+/***************************************************************************
+ * Repetition Count Test
+ *
+ * The LRNG uses an enhanced version of the Repetition Count Test
+ * (RCT) specified in SP800-90B section 4.4.1. Instead of counting identical
+ * back-to-back values, the input to the RCT is the counting of the stuck
+ * values while filling the entropy pool.
+ *
+ * The RCT is applied with an alpha of 2^-30 compliant to FIPS 140-2 IG 9.8.
+ *
+ * During the counting operation, the LRNG always calculates the RCT
+ * cut-off value of C. If that value exceeds the allowed cut-off value,
+ * the LRNG will invalidate all entropy for the entropy pool which implies
+ * that no data can be extracted from the entropy pool unless new entropy
+ * is received.
+ ***************************************************************************/
+
+/*
+ * Hot code path - Insert data for Repetition Count Test
+ *
+ * @health: Reference to health information
+ * @stuck: Decision of stuck test
+ */
+static inline void lrng_rct(struct lrng_health *health, int stuck)
+{
+	struct lrng_rct *rct = &health->rct;
+
+	if (!lrng_sp80090b_health_requested())
+		return;
+
+	if (stuck) {
+		u32 rct_count = atomic_add_return_relaxed(1, &rct->rct_count);
+
+		pr_debug("RCT count: %u\n", rct_count);
+
+		/*
+		 * The cutoff value is based on the following consideration:
+		 * alpha = 2^-30 as recommended in FIPS 140-2 IG 9.8.
+		 * In addition, we imply an entropy value H of 1 bit as this
+		 * is the minimum entropy required to provide full entropy.
+		 *
+		 * Note, rct_count (which equals to value B in the
+		 * pseudo code of SP800-90B section 4.4.1) starts with zero.
+		 * Hence we need to subtract one from the cutoff value as
+		 * calculated following SP800-90B.
+		 */
+		if (rct_count >= CONFIG_LRNG_RCT_CUTOFF) {
+			atomic_set(&rct->rct_count, 0);
+
+			/*
+			 * APT must start anew as we consider all previously
+			 * recorded data to contain no entropy.
+			 */
+			lrng_apt_restart(health);
+
+			lrng_sp80090b_failure(health);
+		}
+	} else {
+		atomic_set(&rct->rct_count, 0);
+	}
+}
+
+/***************************************************************************
+ * Stuck Test
+ *
+ * Checking the:
+ *      1st derivative of the event occurrence (time delta)
+ *      2nd derivative of the event occurrence (delta of time deltas)
+ *      3rd derivative of the event occurrence (delta of delta of time deltas)
+ *
+ * All values must always be non-zero. The stuck test is only valid disabled if
+ * high-resolution time stamps are identified after initialization.
+ ***************************************************************************/
+
+static inline u32 lrng_delta(u32 prev, u32 next)
+{
+	/*
+	 * Note that this (unsigned) subtraction does yield the correct value
+	 * in the wraparound-case, i.e. when next < prev.
+	 */
+	return (next - prev);
+}
+
+/*
+ * Hot code path
+ *
+ * @health: Reference to health information
+ * @now: Event time
+ * @return: 0 event occurrence not stuck (good time stamp)
+ *	    != 0 event occurrence stuck (reject time stamp)
+ */
+static inline int lrng_irq_stuck(struct lrng_stuck_test *stuck, u32 now_time)
+{
+	u32 delta = lrng_delta(stuck->last_time, now_time);
+	u32 delta2 = lrng_delta(stuck->last_delta, delta);
+	u32 delta3 = lrng_delta(stuck->last_delta2, delta2);
+
+	stuck->last_time = now_time;
+	stuck->last_delta = delta;
+	stuck->last_delta2 = delta2;
+
+	if (!delta || !delta2 || !delta3)
+		return 1;
+
+	return 0;
+}
+
+/***************************************************************************
+ * Health test interfaces
+ ***************************************************************************/
+
+/*
+ * Disable all health tests
+ */
+void lrng_health_disable(void)
+{
+	struct lrng_health *health = &lrng_health;
+
+	health->health_test_enabled = false;
+
+	if (lrng_sp80090b_health_requested())
+		pr_warn("SP800-90B compliance requested but the Linux RNG is NOT SP800-90B compliant\n");
+}
+
+/*
+ * Hot code path - Perform health test on time stamp received from an event
+ *
+ * @now_time Time stamp
+ */
+enum lrng_health_res lrng_health_test(u32 now_time)
+{
+	struct lrng_health *health = &lrng_health;
+	struct lrng_stuck_test *stuck_test = this_cpu_ptr(&lrng_stuck_test);
+	int stuck;
+
+	if (!health->health_test_enabled)
+		return lrng_health_pass;
+
+	lrng_apt_insert(health, now_time);
+
+	stuck = lrng_irq_stuck(stuck_test, now_time);
+	lrng_rct(health, stuck);
+	if (stuck) {
+		/* SP800-90B disallows using a failing health test time stamp */
+		return lrng_sp80090b_health_requested() ?
+			lrng_health_fail_drop : lrng_health_fail_use;
+	}
+
+	return lrng_health_pass;
+}
-- 
2.26.2

[PATCH v36 12/13] LRNG - add interface for gathering of raw entropy

[Stephan Müller]

The test interface allows a privileged process to capture the raw
unconditioned noise that is collected by the LRNG for statistical
analysis. Such testing allows the analysis how much entropy
the interrupt noise source provides on a given platform.
Extracted noise data is not used to seed the LRNG. This
is a test interface and not appropriate for production systems.
Yet, the interface is considered to be sufficiently secured for
production systems.

Access to the data is given through the lrng_raw debugfs file. The
data buffer should be multiples of sizeof(u32) to fill the entire
buffer. Using the option lrng_testing.boot_test=1 the raw noise of
the first 1000 entropy events since boot can be sampled.

This test interface allows generating the data required for
analysis whether the LRNG is in compliance with SP800-90B
sections 3.1.3 and 3.1.4.

In addition, the test interface allows gathering of the concatenated raw
entropy data to verify that the concatenation works appropriately.
This includes sampling of the following raw data:

* high-resolution time stamp

* Jiffies

* IRQ number

* IRQ flags

* return instruction pointer

* interrupt register state

* array logic batching the high-resolution time stamp

Also, a testing interface to support ACVT of the hash implementation
is provided. The reason why only hash testing is supported (as
opposed to also provide testing for the DRNG) is the fact that the
LRNG software hash implementation contains glue code that may
warrant testing in addition to the testing of the software ciphers
via the kernel crypto API. Also, for testing the CTR-DRBG, the
underlying AES implementation would need to be tested. However,
such AES test interface cannot be provided by the LRNG as it has no
means to access the AES operation.

Finally, the execution duration for processing a time stamp can be
obtained with the LRNG raw entropy interface.

If a test interface is not compiled, its code is a noop which has no
impact on the performance.

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 drivers/char/lrng/Kconfig        | 150 +++++++
 drivers/char/lrng/Makefile       |   1 +
 drivers/char/lrng/lrng_testing.c | 689 +++++++++++++++++++++++++++++++
 3 files changed, 840 insertions(+)
 create mode 100644 drivers/char/lrng/lrng_testing.c

diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index a059da0b2f5d..bb785bc61abb 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -174,4 +174,154 @@ config LRNG_APT_CUTOFF
 	default 325 if !LRNG_APT_BROKEN
 	default 32 if LRNG_APT_BROKEN
 
+menuconfig LRNG_TESTING_MENU
+	bool "LRNG testing interfaces"
+	depends on DEBUG_FS
+	help
+	  Enable one or more of the following test interfaces.
+
+	  If unsure, say N.
+
+if LRNG_TESTING_MENU
+
+config LRNG_RAW_HIRES_ENTROPY
+	bool "Enable entropy test interface to hires timer noise source"
+	default y
+	help
+	  The test interface allows a privileged process to capture
+	  the raw unconditioned high resolution time stamp noise that
+	  is collected by the LRNG for statistical analysis. Extracted
+	  noise data is not used to seed the LRNG.
+
+	  The raw noise data can be obtained using the lrng_raw_hires
+	  debugfs file. Using the option lrng_testing.boot_raw_hires_test=1
+	  the raw noise of the first 1000 entropy events since boot
+	  can be sampled.
+
+config LRNG_RAW_JIFFIES_ENTROPY
+	bool "Enable entropy test interface to Jiffies noise source"
+	help
+	  The test interface allows a privileged process to capture
+	  the raw unconditioned Jiffies that is collected by
+	  the LRNG for statistical analysis. This data is used for
+	  seeding the LRNG if a high-resolution time stamp is not
+	  available. If a high-resolution time stamp is detected,
+	  the Jiffies value is not collected by the LRNG and no
+	  data is provided via the test interface. Extracted noise
+	  data is not used to seed the random number generator.
+
+	  The raw noise data can be obtained using the lrng_raw_jiffies
+	  debugfs file. Using the option lrng_testing.boot_raw_jiffies_test=1
+	  the raw noise of the first 1000 entropy events since boot
+	  can be sampled.
+
+config LRNG_RAW_IRQ_ENTROPY
+	bool "Enable entropy test interface to IRQ number noise source"
+	help
+	  The test interface allows a privileged process to capture
+	  the raw unconditioned interrupt number that is collected by
+	  the LRNG for statistical analysis. This data is used for
+	  seeding the random32 PRNG external to the LRNG if a
+	  high-resolution time stamp is available or it will be used to
+	  seed the LRNG otherwise. Extracted noise data is not used to
+	  seed the random number generator.
+
+	  The raw noise data can be obtained using the lrng_raw_irq
+	  debugfs file. Using the option lrng_testing.boot_raw_irq_test=1
+	  the raw noise of the first 1000 entropy events since boot
+	  can be sampled.
+
+config LRNG_RAW_IRQFLAGS_ENTROPY
+	bool "Enable entropy test interface to IRQ flags noise source"
+	help
+	  The test interface allows a privileged process to capture
+	  the raw unconditioned interrupt flags that is collected by
+	  the LRNG for statistical analysis. This data is used for
+	  seeding the random32 PRNG external to the LRNG if a
+	  high-resolution time stamp is available or it will be used to
+	  seed the LRNG otherwise. Extracted noise data is not used to
+	  seed the random number generator.
+
+	  The raw noise data can be obtained using the lrng_raw_irqflags
+	  debugfs file. Using the option lrng_testing.boot_raw_irqflags_test=1
+	  the raw noise of the first 1000 entropy events since boot
+	  can be sampled.
+
+config LRNG_RAW_RETIP_ENTROPY
+	bool "Enable entropy test interface to RETIP value noise source"
+	help
+	  The test interface allows a privileged process to capture
+	  the raw unconditioned return instruction pointer value
+	  that is collected by the LRNG for statistical analysis.
+	  This data is used for seeding the random32 PRNG external
+	  to the LRNG if a high-resolution time stamp is available or
+	  it will be used to seed the LRNG otherwise. Extracted noise
+	  data is not used to seed the random number generator.
+
+	  The raw noise data can be obtained using the lrng_raw_retip
+	  debugfs file. Using the option lrng_testing.boot_raw_retip_test=1
+	  the raw noise of the first 1000 entropy events since boot
+	  can be sampled.
+
+config LRNG_RAW_REGS_ENTROPY
+	bool "Enable entropy test interface to IRQ register value noise source"
+	help
+	  The test interface allows a privileged process to capture
+	  the raw unconditioned interrupt register value that is
+	  collected by the LRNG for statistical analysis. Extracted noise
+	  data is not used to seed the random number generator.
+
+	  The raw noise data can be obtained using the lrng_raw_regs
+	  debugfs file. Using the option lrng_testing.boot_raw_regs_test=1
+	  the raw noise of the first 1000 entropy events since boot
+	  can be sampled.
+
+config LRNG_RAW_ARRAY
+	bool "Enable test interface to LRNG raw entropy storage array"
+	help
+	  The test interface allows a privileged process to capture
+	  the raw noise data that is collected by the LRNG
+	  in the per-CPU array for statistical analysis. The purpose
+	  of this interface is to verify that the array handling code
+	  truly only concatenates data and provides the same entropy
+	  rate as the raw unconditioned noise source when assessing
+	  the collected data byte-wise.
+
+	  The data can be obtained using the lrng_raw_array debugfs
+	  file. Using the option lrng_testing.boot_raw_array=1
+	  the raw noise of the first 1000 entropy events since boot
+	  can be sampled.
+
+config LRNG_IRQ_PERF
+	bool "Enable LRNG interrupt performance monitor"
+	help
+	  With this option, the performance monitor of the LRNG
+	  interrupt handling code is enabled. The file provides
+	  the execution time of the interrupt handler in
+	  cycles.
+
+	  The interrupt performance data can be obtained using
+	  the lrng_irq_perf debugfs file. Using the option
+	  lrng_testing.boot_irq_perf=1 the performance data of
+	  the first 1000 entropy events since boot can be sampled.
+
+config LRNG_ACVT_HASH
+	bool "Enable LRNG ACVT Hash interface"
+	help
+	  With this option, the LRNG built-in hash function used for
+	  auxiliary pool management and prior to switching the
+	  cryptographic backends is made available for ACVT. The
+	  interface allows writing of the data to be hashed
+	  into the interface. The read operation triggers the hash
+	  operation to generate message digest.
+
+	  The ACVT interface is available with the lrng_acvt_hash
+	  debugfs file.
+
+config LRNG_TESTING
+	bool
+	default y if (LRNG_RAW_HIRES_ENTROPY || LRNG_RAW_JIFFIES_ENTROPY ||LRNG_RAW_IRQ_ENTROPY || LRNG_RAW_IRQFLAGS_ENTROPY || LRNG_RAW_RETIP_ENTROPY || LRNG_RAW_REGS_ENTROPY || LRNG_RAW_ARRAY || LRNG_IRQ_PERF || LRNG_ACVT_HASH)
+
+endif #LRNG_TESTING_MENU
+
 endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 3c8637befb42..532501b38a00 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -16,3 +16,4 @@ obj-$(CONFIG_LRNG_DRBG)		+= lrng_drbg.o
 obj-$(CONFIG_LRNG_KCAPI)	+= lrng_kcapi.o
 obj-$(CONFIG_LRNG_JENT)		+= lrng_jent.o
 obj-$(CONFIG_LRNG_HEALTH_TESTS)	+= lrng_health.o
+obj-$(CONFIG_LRNG_TESTING)	+= lrng_testing.o
diff --git a/drivers/char/lrng/lrng_testing.c b/drivers/char/lrng/lrng_testing.c
new file mode 100644
index 000000000000..aafdff6304f1
--- /dev/null
+++ b/drivers/char/lrng/lrng_testing.c
@@ -0,0 +1,689 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Linux Random Number Generator (LRNG) testing interfaces
+ *
+ * Copyright (C) 2019 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/atomic.h>
+#include <linux/bug.h>
+#include <linux/debugfs.h>
+#include <linux/lrng.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/sched/signal.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/uaccess.h>
+#include <linux/workqueue.h>
+#include <asm/errno.h>
+
+#include "lrng_internal.h"
+
+#define LRNG_TESTING_RINGBUFFER_SIZE	1024
+#define LRNG_TESTING_RINGBUFFER_MASK	(LRNG_TESTING_RINGBUFFER_SIZE - 1)
+
+struct lrng_testing {
+	u32 lrng_testing_rb[LRNG_TESTING_RINGBUFFER_SIZE];
+	u32 rb_reader;
+	u32 rb_writer;
+	atomic_t lrng_testing_enabled;
+	spinlock_t lock;
+	wait_queue_head_t read_wait;
+};
+
+/*************************** Generic Data Handling ****************************/
+
+/*
+ * boot variable:
+ * 0 ==> No boot test, gathering of runtime data allowed
+ * 1 ==> Boot test enabled and ready for collecting data, gathering runtime
+ *	 data is disabled
+ * 2 ==> Boot test completed and disabled, gathering of runtime data is
+ *	 disabled
+ */
+
+static inline void lrng_testing_reset(struct lrng_testing *data)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&data->lock, flags);
+	data->rb_reader = 0;
+	data->rb_writer = 0;
+	spin_unlock_irqrestore(&data->lock, flags);
+}
+
+static inline void lrng_testing_init(struct lrng_testing *data, u32 boot)
+{
+	/*
+	 * The boot time testing implies we have a running test. If the
+	 * caller wants to clear it, he has to unset the boot_test flag
+	 * at runtime via sysfs to enable regular runtime testing
+	 */
+	if (boot)
+		return;
+
+	lrng_testing_reset(data);
+	atomic_set(&data->lrng_testing_enabled, 1);
+	pr_warn("Enabling data collection\n");
+}
+
+static inline void lrng_testing_fini(struct lrng_testing *data, u32 boot)
+{
+	/* If we have boot data, we do not reset yet to allow data to be read */
+	if (boot)
+		return;
+
+	atomic_set(&data->lrng_testing_enabled, 0);
+	lrng_testing_reset(data);
+	pr_warn("Disabling data collection\n");
+}
+
+static inline bool lrng_testing_store(struct lrng_testing *data, u32 value,
+				      u32 *boot)
+{
+	unsigned long flags;
+
+	if (!atomic_read(&data->lrng_testing_enabled) && (*boot != 1))
+		return false;
+
+	spin_lock_irqsave(&data->lock, flags);
+
+	/*
+	 * Disable entropy testing for boot time testing after ring buffer
+	 * is filled.
+	 */
+	if (*boot) {
+		if (data->rb_writer > LRNG_TESTING_RINGBUFFER_SIZE) {
+			*boot = 2;
+			pr_warn_once("One time data collection test disabled\n");
+			spin_unlock_irqrestore(&data->lock, flags);
+			return false;
+		}
+
+		if (data->rb_writer == 1)
+			pr_warn("One time data collection test enabled\n");
+	}
+
+	data->lrng_testing_rb[data->rb_writer & LRNG_TESTING_RINGBUFFER_MASK] =
+									value;
+	data->rb_writer++;
+
+	spin_unlock_irqrestore(&data->lock, flags);
+
+	if (wq_has_sleeper(&data->read_wait))
+		wake_up_interruptible(&data->read_wait);
+
+	return true;
+}
+
+static inline bool lrng_testing_have_data(struct lrng_testing *data)
+{
+	return ((data->rb_writer & LRNG_TESTING_RINGBUFFER_MASK) !=
+		 (data->rb_reader & LRNG_TESTING_RINGBUFFER_MASK));
+}
+
+static inline int lrng_testing_reader(struct lrng_testing *data, u32 *boot,
+				      u8 *outbuf, u32 outbuflen)
+{
+	unsigned long flags;
+	int collected_data = 0;
+
+	lrng_testing_init(data, *boot);
+
+	while (outbuflen) {
+		spin_lock_irqsave(&data->lock, flags);
+
+		/* We have no data or reached the writer. */
+		if (!data->rb_writer ||
+		    (data->rb_writer == data->rb_reader)) {
+
+			spin_unlock_irqrestore(&data->lock, flags);
+
+			/*
+			 * Now we gathered all boot data, enable regular data
+			 * collection.
+			 */
+			if (*boot) {
+				*boot = 0;
+				goto out;
+			}
+
+			wait_event_interruptible(data->read_wait,
+						 lrng_testing_have_data(data));
+			if (signal_pending(current)) {
+				collected_data = -ERESTARTSYS;
+				goto out;
+			}
+
+			continue;
+		}
+
+		/* We copy out word-wise */
+		if (outbuflen < sizeof(u32)) {
+			spin_unlock_irqrestore(&data->lock, flags);
+			goto out;
+		}
+
+		memcpy(outbuf, &data->lrng_testing_rb[data->rb_reader],
+		       sizeof(u32));
+		data->rb_reader++;
+
+		spin_unlock_irqrestore(&data->lock, flags);
+
+		outbuf += sizeof(u32);
+		outbuflen -= sizeof(u32);
+		collected_data += sizeof(u32);
+	}
+
+out:
+	lrng_testing_fini(data, *boot);
+	return collected_data;
+}
+
+static int lrng_testing_extract_user(struct file *file, char __user *buf,
+				     size_t nbytes, loff_t *ppos,
+				     int (*reader)(u8 *outbuf, u32 outbuflen))
+{
+	u8 *tmp, *tmp_aligned;
+	int ret = 0, large_request = (nbytes > 256);
+
+	if (!nbytes)
+		return 0;
+
+	/*
+	 * The intention of this interface is for collecting at least
+	 * 1000 samples due to the SP800-90B requirements. So, we make no
+	 * effort in avoiding allocating more memory that actually needed
+	 * by the user. Hence, we allocate sufficient memory to always hold
+	 * that amount of data.
+	 */
+	tmp = kmalloc(LRNG_TESTING_RINGBUFFER_SIZE + sizeof(u32), GFP_KERNEL);
+	if (!tmp)
+		return -ENOMEM;
+
+	tmp_aligned = PTR_ALIGN(tmp, sizeof(u32));
+
+	while (nbytes) {
+		int i;
+
+		if (large_request && need_resched()) {
+			if (signal_pending(current)) {
+				if (ret == 0)
+					ret = -ERESTARTSYS;
+				break;
+			}
+			schedule();
+		}
+
+		i = min_t(int, nbytes, LRNG_TESTING_RINGBUFFER_SIZE);
+		i = reader(tmp_aligned, i);
+		if (i <= 0) {
+			if (i < 0)
+				ret = i;
+			break;
+		}
+		if (copy_to_user(buf, tmp_aligned, i)) {
+			ret = -EFAULT;
+			break;
+		}
+
+		nbytes -= i;
+		buf += i;
+		ret += i;
+	}
+
+	kfree_sensitive(tmp);
+
+	if (ret > 0)
+		*ppos += ret;
+
+	return ret;
+}
+
+/************** Raw High-Resolution Timer Entropy Data Handling ***************/
+
+#ifdef CONFIG_LRNG_RAW_HIRES_ENTROPY
+
+static u32 boot_raw_hires_test = 0;
+module_param(boot_raw_hires_test, uint, 0644);
+MODULE_PARM_DESC(boot_raw_hires_test, "Enable gathering boot time high resolution timer entropy of the first entropy events");
+
+static struct lrng_testing lrng_raw_hires = {
+	.rb_reader = 0,
+	.rb_writer = 0,
+	.lock      = __SPIN_LOCK_UNLOCKED(lrng_raw_hires.lock),
+	.read_wait = __WAIT_QUEUE_HEAD_INITIALIZER(lrng_raw_hires.read_wait)
+};
+
+bool lrng_raw_hires_entropy_store(u32 value)
+{
+	return lrng_testing_store(&lrng_raw_hires, value, &boot_raw_hires_test);
+}
+
+static int lrng_raw_hires_entropy_reader(u8 *outbuf, u32 outbuflen)
+{
+	return lrng_testing_reader(&lrng_raw_hires, &boot_raw_hires_test,
+				   outbuf, outbuflen);
+}
+
+static ssize_t lrng_raw_hires_read(struct file *file, char __user *to,
+				   size_t count, loff_t *ppos)
+{
+	return lrng_testing_extract_user(file, to, count, ppos,
+					 lrng_raw_hires_entropy_reader);
+}
+
+static const struct file_operations lrng_raw_hires_fops = {
+	.owner = THIS_MODULE,
+	.read = lrng_raw_hires_read,
+};
+
+#endif /* CONFIG_LRNG_RAW_HIRES_ENTROPY */
+
+/********************* Raw Jiffies Entropy Data Handling **********************/
+
+#ifdef CONFIG_LRNG_RAW_JIFFIES_ENTROPY
+
+static u32 boot_raw_jiffies_test = 0;
+module_param(boot_raw_jiffies_test, uint, 0644);
+MODULE_PARM_DESC(boot_raw_jiffies_test, "Enable gathering boot time high resolution timer entropy of the first entropy events");
+
+static struct lrng_testing lrng_raw_jiffies = {
+	.rb_reader = 0,
+	.rb_writer = 0,
+	.lock      = __SPIN_LOCK_UNLOCKED(lrng_raw_jiffies.lock),
+	.read_wait = __WAIT_QUEUE_HEAD_INITIALIZER(lrng_raw_jiffies.read_wait)
+};
+
+bool lrng_raw_jiffies_entropy_store(u32 value)
+{
+	return lrng_testing_store(&lrng_raw_jiffies, value,
+				  &boot_raw_jiffies_test);
+}
+
+static int lrng_raw_jiffies_entropy_reader(u8 *outbuf, u32 outbuflen)
+{
+	return lrng_testing_reader(&lrng_raw_jiffies, &boot_raw_jiffies_test,
+				   outbuf, outbuflen);
+}
+
+static ssize_t lrng_raw_jiffies_read(struct file *file, char __user *to,
+				   size_t count, loff_t *ppos)
+{
+	return lrng_testing_extract_user(file, to, count, ppos,
+					 lrng_raw_jiffies_entropy_reader);
+}
+
+static const struct file_operations lrng_raw_jiffies_fops = {
+	.owner = THIS_MODULE,
+	.read = lrng_raw_jiffies_read,
+};
+
+#endif /* CONFIG_LRNG_RAW_JIFFIES_ENTROPY */
+
+/************************** Raw IRQ Data Handling ****************************/
+
+#ifdef CONFIG_LRNG_RAW_IRQ_ENTROPY
+
+static u32 boot_raw_irq_test = 0;
+module_param(boot_raw_irq_test, uint, 0644);
+MODULE_PARM_DESC(boot_raw_irq_test, "Enable gathering boot time entropy of the first IRQ entropy events");
+
+static struct lrng_testing lrng_raw_irq = {
+	.rb_reader = 0,
+	.rb_writer = 0,
+	.lock      = __SPIN_LOCK_UNLOCKED(lrng_raw_irq.lock),
+	.read_wait = __WAIT_QUEUE_HEAD_INITIALIZER(lrng_raw_irq.read_wait)
+};
+
+bool lrng_raw_irq_entropy_store(u32 value)
+{
+	return lrng_testing_store(&lrng_raw_irq, value, &boot_raw_irq_test);
+}
+
+static int lrng_raw_irq_entropy_reader(u8 *outbuf, u32 outbuflen)
+{
+	return lrng_testing_reader(&lrng_raw_irq, &boot_raw_irq_test, outbuf,
+				   outbuflen);
+}
+
+static ssize_t lrng_raw_irq_read(struct file *file, char __user *to,
+				 size_t count, loff_t *ppos)
+{
+	return lrng_testing_extract_user(file, to, count, ppos,
+					 lrng_raw_irq_entropy_reader);
+}
+
+static const struct file_operations lrng_raw_irq_fops = {
+	.owner = THIS_MODULE,
+	.read = lrng_raw_irq_read,
+};
+
+#endif /* CONFIG_LRNG_RAW_IRQ_ENTROPY */
+
+/************************ Raw IRQFLAGS Data Handling **************************/
+
+#ifdef CONFIG_LRNG_RAW_IRQFLAGS_ENTROPY
+
+static u32 boot_raw_irqflags_test = 0;
+module_param(boot_raw_irqflags_test, uint, 0644);
+MODULE_PARM_DESC(boot_raw_irqflags_test, "Enable gathering boot time entropy of the first IRQ flags entropy events");
+
+static struct lrng_testing lrng_raw_irqflags = {
+	.rb_reader = 0,
+	.rb_writer = 0,
+	.lock      = __SPIN_LOCK_UNLOCKED(lrng_raw_irqflags.lock),
+	.read_wait = __WAIT_QUEUE_HEAD_INITIALIZER(lrng_raw_irqflags.read_wait)
+};
+
+bool lrng_raw_irqflags_entropy_store(u32 value)
+{
+	return lrng_testing_store(&lrng_raw_irqflags, value,
+				  &boot_raw_irqflags_test);
+}
+
+static int lrng_raw_irqflags_entropy_reader(u8 *outbuf, u32 outbuflen)
+{
+	return lrng_testing_reader(&lrng_raw_irqflags, &boot_raw_irqflags_test,
+				   outbuf, outbuflen);
+}
+
+static ssize_t lrng_raw_irqflags_read(struct file *file, char __user *to,
+				      size_t count, loff_t *ppos)
+{
+	return lrng_testing_extract_user(file, to, count, ppos,
+					 lrng_raw_irqflags_entropy_reader);
+}
+
+static const struct file_operations lrng_raw_irqflags_fops = {
+	.owner = THIS_MODULE,
+	.read = lrng_raw_irqflags_read,
+};
+
+#endif /* CONFIG_LRNG_RAW_IRQFLAGS_ENTROPY */
+
+/************************ Raw _RET_IP_ Data Handling **************************/
+
+#ifdef CONFIG_LRNG_RAW_RETIP_ENTROPY
+
+static u32 boot_raw_retip_test = 0;
+module_param(boot_raw_retip_test, uint, 0644);
+MODULE_PARM_DESC(boot_raw_retip_test, "Enable gathering boot time entropy of the first return instruction pointer entropy events");
+
+static struct lrng_testing lrng_raw_retip = {
+	.rb_reader = 0,
+	.rb_writer = 0,
+	.lock      = __SPIN_LOCK_UNLOCKED(lrng_raw_retip.lock),
+	.read_wait = __WAIT_QUEUE_HEAD_INITIALIZER(lrng_raw_retip.read_wait)
+};
+
+bool lrng_raw_retip_entropy_store(u32 value)
+{
+	return lrng_testing_store(&lrng_raw_retip, value, &boot_raw_retip_test);
+}
+
+static int lrng_raw_retip_entropy_reader(u8 *outbuf, u32 outbuflen)
+{
+	return lrng_testing_reader(&lrng_raw_retip, &boot_raw_retip_test,
+				   outbuf, outbuflen);
+}
+
+static ssize_t lrng_raw_retip_read(struct file *file, char __user *to,
+				   size_t count, loff_t *ppos)
+{
+	return lrng_testing_extract_user(file, to, count, ppos,
+					 lrng_raw_retip_entropy_reader);
+}
+
+static const struct file_operations lrng_raw_retip_fops = {
+	.owner = THIS_MODULE,
+	.read = lrng_raw_retip_read,
+};
+
+#endif /* CONFIG_LRNG_RAW_RETIP_ENTROPY */
+
+/********************** Raw IRQ register Data Handling ************************/
+
+#ifdef CONFIG_LRNG_RAW_REGS_ENTROPY
+
+static u32 boot_raw_regs_test = 0;
+module_param(boot_raw_regs_test, uint, 0644);
+MODULE_PARM_DESC(boot_raw_regs_test, "Enable gathering boot time entropy of the first interrupt register entropy events");
+
+static struct lrng_testing lrng_raw_regs = {
+	.rb_reader = 0,
+	.rb_writer = 0,
+	.lock      = __SPIN_LOCK_UNLOCKED(lrng_raw_regs.lock),
+	.read_wait = __WAIT_QUEUE_HEAD_INITIALIZER(lrng_raw_regs.read_wait)
+};
+
+bool lrng_raw_regs_entropy_store(u32 value)
+{
+	return lrng_testing_store(&lrng_raw_regs, value, &boot_raw_regs_test);
+}
+
+static int lrng_raw_regs_entropy_reader(u8 *outbuf, u32 outbuflen)
+{
+	return lrng_testing_reader(&lrng_raw_regs, &boot_raw_regs_test,
+				   outbuf, outbuflen);
+}
+
+static ssize_t lrng_raw_regs_read(struct file *file, char __user *to,
+				  size_t count, loff_t *ppos)
+{
+	return lrng_testing_extract_user(file, to, count, ppos,
+					 lrng_raw_regs_entropy_reader);
+}
+
+static const struct file_operations lrng_raw_regs_fops = {
+	.owner = THIS_MODULE,
+	.read = lrng_raw_regs_read,
+};
+
+#endif /* CONFIG_LRNG_RAW_REGS_ENTROPY */
+
+/********************** Raw Entropy Array Data Handling ***********************/
+
+#ifdef CONFIG_LRNG_RAW_ARRAY
+
+static u32 boot_raw_array = 0;
+module_param(boot_raw_array, uint, 0644);
+MODULE_PARM_DESC(boot_raw_array, "Enable gathering boot time raw noise array data of the first entropy events");
+
+static struct lrng_testing lrng_raw_array = {
+	.rb_reader = 0,
+	.rb_writer = 0,
+	.lock      = __SPIN_LOCK_UNLOCKED(lrng_raw_array.lock),
+	.read_wait = __WAIT_QUEUE_HEAD_INITIALIZER(lrng_raw_array.read_wait)
+};
+
+bool lrng_raw_array_entropy_store(u32 value)
+{
+	return lrng_testing_store(&lrng_raw_array, value, &boot_raw_array);
+}
+
+static int lrng_raw_array_entropy_reader(u8 *outbuf, u32 outbuflen)
+{
+	return lrng_testing_reader(&lrng_raw_array, &boot_raw_array, outbuf,
+				   outbuflen);
+}
+
+static ssize_t lrng_raw_array_read(struct file *file, char __user *to,
+				   size_t count, loff_t *ppos)
+{
+	return lrng_testing_extract_user(file, to, count, ppos,
+					 lrng_raw_array_entropy_reader);
+}
+
+static const struct file_operations lrng_raw_array_fops = {
+	.owner = THIS_MODULE,
+	.read = lrng_raw_array_read,
+};
+
+#endif /* CONFIG_LRNG_RAW_ARRAY */
+
+/******************** Interrupt Performance Data Handling *********************/
+
+#ifdef CONFIG_LRNG_IRQ_PERF
+
+static u32 boot_irq_perf = 0;
+module_param(boot_irq_perf, uint, 0644);
+MODULE_PARM_DESC(boot_irq_perf, "Enable gathering boot time interrupt performance data of the first entropy events");
+
+static struct lrng_testing lrng_irq_perf = {
+	.rb_reader = 0,
+	.rb_writer = 0,
+	.lock      = __SPIN_LOCK_UNLOCKED(lrng_irq_perf.lock),
+	.read_wait = __WAIT_QUEUE_HEAD_INITIALIZER(lrng_irq_perf.read_wait)
+};
+
+bool lrng_perf_time(u32 start)
+{
+	return lrng_testing_store(&lrng_irq_perf, random_get_entropy() - start,
+				  &boot_irq_perf);
+}
+
+static int lrng_irq_perf_reader(u8 *outbuf, u32 outbuflen)
+{
+	return lrng_testing_reader(&lrng_irq_perf, &boot_irq_perf, outbuf,
+				   outbuflen);
+}
+
+static ssize_t lrng_irq_perf_read(struct file *file, char __user *to,
+				  size_t count, loff_t *ppos)
+{
+	return lrng_testing_extract_user(file, to, count, ppos,
+					 lrng_irq_perf_reader);
+}
+
+static const struct file_operations lrng_irq_perf_fops = {
+	.owner = THIS_MODULE,
+	.read = lrng_irq_perf_read,
+};
+
+#endif /* CONFIG_LRNG_IRQ_PERF */
+
+/*********************************** ACVT ************************************/
+
+#ifdef CONFIG_LRNG_ACVT_HASH
+
+/* maximum amount of data to be hashed as defined by ACVP */
+#define LRNG_ACVT_MAX_SHA_MSG	(65536 >> 3)
+
+/*
+ * As we use static variables to store the data, it is clear that the
+ * test interface is only able to handle single threaded testing. This is
+ * considered to be sufficient for testing. If multi-threaded use of the
+ * ACVT test interface would be performed, the caller would get garbage
+ * but the kernel operation is unaffected by this.
+ */
+static u8 lrng_acvt_hash_data[LRNG_ACVT_MAX_SHA_MSG]
+						__aligned(LRNG_KCAPI_ALIGN);
+static atomic_t lrng_acvt_hash_data_size = ATOMIC_INIT(0);
+static u8 lrng_acvt_hash_digest[LRNG_ATOMIC_DIGEST_SIZE];
+
+static ssize_t lrng_acvt_hash_write(struct file *file, const char __user *buf,
+				    size_t nbytes, loff_t *ppos)
+{
+	if (nbytes > LRNG_ACVT_MAX_SHA_MSG)
+		return -EINVAL;
+
+	atomic_set(&lrng_acvt_hash_data_size, (int)nbytes);
+
+	return simple_write_to_buffer(lrng_acvt_hash_data,
+				      LRNG_ACVT_MAX_SHA_MSG, ppos, buf, nbytes);
+}
+
+static ssize_t lrng_acvt_hash_read(struct file *file, char __user *to,
+				   size_t count, loff_t *ppos)
+{
+	SHASH_DESC_ON_STACK(shash, NULL);
+	const struct lrng_crypto_cb *crypto_cb = &lrng_cc20_crypto_cb;
+	ssize_t ret;
+
+	if (count > LRNG_ATOMIC_DIGEST_SIZE)
+		return -EINVAL;
+
+	ret = crypto_cb->lrng_hash_init(shash, NULL) ?:
+	      crypto_cb->lrng_hash_update(shash, lrng_acvt_hash_data,
+				atomic_read_u32(&lrng_acvt_hash_data_size)) ?:
+	      crypto_cb->lrng_hash_final(shash, lrng_acvt_hash_digest);
+	if (ret)
+		return ret;
+
+	return simple_read_from_buffer(to, count, ppos, lrng_acvt_hash_digest,
+				       sizeof(lrng_acvt_hash_digest));
+}
+
+static const struct file_operations lrng_acvt_hash_fops = {
+	.owner = THIS_MODULE,
+	.open = simple_open,
+	.llseek = default_llseek,
+	.read = lrng_acvt_hash_read,
+	.write = lrng_acvt_hash_write,
+};
+
+#endif /* CONFIG_LRNG_ACVT_DRNG */
+
+/**************************************************************************
+ * Debugfs interface
+ **************************************************************************/
+
+static int __init lrng_raw_init(void)
+{
+	struct dentry *lrng_raw_debugfs_root;
+
+	lrng_raw_debugfs_root = debugfs_create_dir(KBUILD_MODNAME, NULL);
+
+#ifdef CONFIG_LRNG_RAW_HIRES_ENTROPY
+	debugfs_create_file_unsafe("lrng_raw_hires", 0400,
+				   lrng_raw_debugfs_root, NULL,
+				   &lrng_raw_hires_fops);
+#endif
+#ifdef CONFIG_LRNG_RAW_JIFFIES_ENTROPY
+	debugfs_create_file_unsafe("lrng_raw_jiffies", 0400,
+				   lrng_raw_debugfs_root, NULL,
+				   &lrng_raw_jiffies_fops);
+#endif
+#ifdef CONFIG_LRNG_RAW_IRQ_ENTROPY
+	debugfs_create_file_unsafe("lrng_raw_irq", 0400, lrng_raw_debugfs_root,
+				   NULL, &lrng_raw_irq_fops);
+#endif
+#ifdef CONFIG_LRNG_RAW_IRQFLAGS_ENTROPY
+	debugfs_create_file_unsafe("lrng_raw_irqflags", 0400,
+				   lrng_raw_debugfs_root, NULL,
+				   &lrng_raw_irqflags_fops);
+#endif
+#ifdef CONFIG_LRNG_RAW_RETIP_ENTROPY
+	debugfs_create_file_unsafe("lrng_raw_retip", 0400,
+				   lrng_raw_debugfs_root, NULL,
+				   &lrng_raw_retip_fops);
+#endif
+#ifdef CONFIG_LRNG_RAW_REGS_ENTROPY
+	debugfs_create_file_unsafe("lrng_raw_regs", 0400,
+				   lrng_raw_debugfs_root, NULL,
+				   &lrng_raw_regs_fops);
+#endif
+#ifdef CONFIG_LRNG_RAW_ARRAY
+	debugfs_create_file_unsafe("lrng_raw_array", 0400,
+				   lrng_raw_debugfs_root, NULL,
+				   &lrng_raw_array_fops);
+#endif
+#ifdef CONFIG_LRNG_IRQ_PERF
+	debugfs_create_file_unsafe("lrng_irq_perf", 0400, lrng_raw_debugfs_root,
+				   NULL, &lrng_irq_perf_fops);
+#endif
+#ifdef CONFIG_LRNG_ACVT_HASH
+	debugfs_create_file_unsafe("lrng_acvt_hash", 0600,
+				   lrng_raw_debugfs_root, NULL,
+				   &lrng_acvt_hash_fops);
+#endif
+
+	return 0;
+}
+
+module_init(lrng_raw_init);
-- 
2.26.2

[PATCH v36 13/13] LRNG - add power-on and runtime self-tests

[Stephan Müller]

Parts of the LRNG are already covered by self-tests, including:

* Self-test of SP800-90A DRBG provided by the Linux kernel crypto API.

* Self-test of the PRNG provided by the Linux kernel crypto API.

* Raw noise source data testing including SP800-90B compliant
  tests when enabling CONFIG_LRNG_HEALTH_TESTS

This patch adds the self-tests for the remaining critical functions of
the LRNG that are essential to maintain entropy and provide
cryptographic strong random numbers. The following self-tests are
implemented:

* Self-test of the time array maintenance. This test verifies whether
the time stamp array management to store multiple values in one integer
implements a concatenation of the data.

* Self-test of the software hash implementation ensures that this
function operates compliant to the FIPS 180-4 specification. The
self-test performs a hash operation of a zeroized per-CPU data array.

* Self-test of the ChaCha20 DRNG is based on the self-tests that are
already present and implemented with the stand-alone user space
ChaCha20 DRNG implementation available at [1]. The self-tests cover
different use cases of the DRNG seeded with known seed data.

The status of the LRNG self-tests is provided with the selftest_status
SysFS file. If the file contains a zero, the self-tests passed. The
value 0xffffffff means that the self-tests were not executed. Any other
value indicates a self-test failure.

The self-test may be compiled to panic the system if the self-test
fails.

All self-tests operate on private state data structures. This implies
that none of the self-tests have any impact on the regular LRNG
operations. This allows the self-tests to be repeated at runtime by
writing anything into the selftest_status SysFS file.

[1] https://www.chronox.de/chacha20.html

CC: Torsten Duwe <duwe@lst.de>
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
CC: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
CC: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
 drivers/char/lrng/Kconfig         |  26 +++
 drivers/char/lrng/Makefile        |   1 +
 drivers/char/lrng/lrng_selftest.c | 344 ++++++++++++++++++++++++++++++
 3 files changed, 371 insertions(+)
 create mode 100644 drivers/char/lrng/lrng_selftest.c

diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index bb785bc61abb..6f180641a5da 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -324,4 +324,30 @@ config LRNG_TESTING
 
 endif #LRNG_TESTING_MENU
 
+config LRNG_SELFTEST
+	bool "Enable power-on and on-demand self-tests"
+	help
+	  The power-on self-tests are executed during boot time
+	  covering the ChaCha20 DRNG, the hash operation used for
+	  processing the entropy pools and the auxiliary pool, and
+	  the time stamp management of the LRNG.
+
+	  The on-demand self-tests are triggered by writing any
+	  value into the SysFS file selftest_status. At the same
+	  time, when reading this file, the test status is
+	  returned. A zero indicates that all tests were executed
+	  successfully.
+
+	  If unsure, say Y.
+
+if LRNG_SELFTEST
+
+config LRNG_SELFTEST_PANIC
+	bool "Panic the kernel upon self-test failure"
+	help
+	  If the option is enabled, the kernel is terminated if an
+	  LRNG power-on self-test failure is detected.
+
+endif # LRNG_SELFTEST
+
 endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 532501b38a00..a633638af991 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -17,3 +17,4 @@ obj-$(CONFIG_LRNG_KCAPI)	+= lrng_kcapi.o
 obj-$(CONFIG_LRNG_JENT)		+= lrng_jent.o
 obj-$(CONFIG_LRNG_HEALTH_TESTS)	+= lrng_health.o
 obj-$(CONFIG_LRNG_TESTING)	+= lrng_testing.o
+obj-$(CONFIG_LRNG_SELFTEST)	+= lrng_selftest.o
diff --git a/drivers/char/lrng/lrng_selftest.c b/drivers/char/lrng/lrng_selftest.c
new file mode 100644
index 000000000000..4c7d124d24a4
--- /dev/null
+++ b/drivers/char/lrng/lrng_selftest.c
@@ -0,0 +1,344 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG power-on and on-demand self-test
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+/*
+ * In addition to the self-tests below, the following LRNG components
+ * are covered with self-tests during regular operation:
+ *
+ * * power-on self-test: SP800-90A DRBG provided by the Linux kernel crypto API
+ * * power-on self-test: PRNG provided by the Linux kernel crypto API
+ * * runtime test: Raw noise source data testing including SP800-90B compliant
+ *		   tests when enabling CONFIG_LRNG_HEALTH_TESTS
+ *
+ * Additional developer tests present with LRNG code:
+ * * SP800-90B APT and RCT test enforcement validation when enabling
+ *   CONFIG_LRNG_APT_BROKEN or CONFIG_LRNG_RCT_BROKEN.
+ * * Collection of raw entropy from the interrupt noise source when enabling
+ *   CONFIG_LRNG_TESTING and pulling the data from the kernel with the provided
+ *   interface.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/module.h>
+#include <linux/lrng.h>
+#include <linux/slab.h>
+
+#include "lrng_chacha20.h"
+#include "lrng_internal.h"
+#include "lrng_sw_noise.h"
+
+#define LRNG_SELFTEST_PASSED		0
+#define LRNG_SEFLTEST_ERROR_TIME	(1 << 0)
+#define LRNG_SEFLTEST_ERROR_CHACHA20	(1 << 1)
+#define LRNG_SEFLTEST_ERROR_HASH	(1 << 2)
+#define LRNG_SELFTEST_NOT_EXECUTED	0xffffffff
+
+static u32 lrng_data_selftest_ptr = 0;
+static u32 lrng_data_selftest[LRNG_DATA_ARRAY_SIZE];
+
+static unsigned int lrng_selftest_status = LRNG_SELFTEST_NOT_EXECUTED;
+
+static inline void lrng_selftest_bswap32(u32 *ptr, u32 words)
+{
+	u32 i;
+
+	/* Byte-swap data which is an LE representation */
+	for (i = 0; i < words; i++) {
+		__le32 *p = (__le32 *)ptr;
+
+		*p = cpu_to_le32(*ptr);
+		ptr++;
+	}
+}
+
+static inline void lrng_data_process_selftest_insert(u32 time)
+{
+	u32 ptr = lrng_data_selftest_ptr++ & LRNG_DATA_WORD_MASK;
+
+	lrng_data_selftest[lrng_data_idx2array(ptr)] |=
+		lrng_data_slot_val(time & LRNG_DATA_SLOTSIZE_MASK,
+				   lrng_data_idx2slot(ptr));
+}
+
+static inline void lrng_data_process_selftest_u32(uint32_t data)
+{
+	uint32_t pre_ptr, ptr, mask;
+
+	/* Increment pointer by number of slots taken for input value */
+	lrng_data_selftest_ptr += LRNG_DATA_SLOTS_PER_UINT;
+
+	/* ptr to current unit */
+	ptr = lrng_data_selftest_ptr;
+	/* ptr to previous unit */
+	pre_ptr = (lrng_data_selftest_ptr - LRNG_DATA_SLOTS_PER_UINT) &
+		  LRNG_DATA_WORD_MASK;
+	ptr &= LRNG_DATA_WORD_MASK;
+
+	/* mask to split data into the two parts for the two units */
+	mask = ((1 << (pre_ptr & (LRNG_DATA_SLOTS_PER_UINT - 1)) *
+		       LRNG_DATA_SLOTSIZE_BITS)) - 1;
+
+	/* MSB of data go into previous unit */
+	lrng_data_selftest[lrng_data_idx2array(pre_ptr)] |= data & ~mask;
+
+	/* LSB of data go into current unit */
+	lrng_data_selftest[lrng_data_idx2array(ptr)] = data & mask;
+}
+
+static unsigned int lrng_data_process_selftest(void)
+{
+	u32 time;
+	u32 idx_zero_compare = (0 << 0) | (1 << 8) | (2 << 16) | (3 << 24);
+	u32 idx_one_compare  = (4 << 0) | (5 << 8) | (6 << 16) | (7 << 24);
+	u32 idx_last_compare =
+		(((LRNG_DATA_NUM_VALUES - 4) & LRNG_DATA_SLOTSIZE_MASK) << 0)  |
+		(((LRNG_DATA_NUM_VALUES - 3) & LRNG_DATA_SLOTSIZE_MASK) << 8)  |
+		(((LRNG_DATA_NUM_VALUES - 2) & LRNG_DATA_SLOTSIZE_MASK) << 16) |
+		(((LRNG_DATA_NUM_VALUES - 1) & LRNG_DATA_SLOTSIZE_MASK) << 24);
+
+	(void)idx_one_compare;
+
+	lrng_data_process_selftest_insert(0);
+	/*
+	 * Note, when using lrng_data_process_u32() on unaligned ptr,
+	 * the first slots will go into next word, and the last slots go
+	 * into the previous word.
+	 */
+	lrng_data_process_selftest_u32((4 << 0) | (1 << 8) | (2 << 16) |
+				       (3 << 24));
+	lrng_data_process_selftest_insert(5);
+	lrng_data_process_selftest_insert(6);
+	lrng_data_process_selftest_insert(7);
+
+	if ((lrng_data_selftest[0] != idx_zero_compare) ||
+	    (lrng_data_selftest[1] != idx_one_compare))
+		goto err;
+
+	/* Reset for next test */
+	lrng_data_selftest[0] = 0;
+	lrng_data_selftest[1] = 0;
+	lrng_data_selftest_ptr = 0;
+
+	for (time = 0; time < LRNG_DATA_NUM_VALUES; time++)
+		lrng_data_process_selftest_insert(time);
+
+	if ((lrng_data_selftest[0] != idx_zero_compare) ||
+	    (lrng_data_selftest[1] != idx_one_compare)  ||
+	    (lrng_data_selftest[LRNG_DATA_ARRAY_SIZE - 1] != idx_last_compare))
+		goto err;
+
+	return LRNG_SELFTEST_PASSED;
+
+err:
+	pr_err("LRNG data array self-test FAILED\n");
+	return LRNG_SEFLTEST_ERROR_TIME;
+}
+
+/* The test vectors are taken from crypto/testmgr.h */
+static unsigned int lrng_hash_selftest(void)
+{
+	SHASH_DESC_ON_STACK(shash, NULL);
+	const struct lrng_crypto_cb *crypto_cb = &lrng_cc20_crypto_cb;
+	static const u8 lrng_hash_selftest_result[] =
+#ifdef CONFIG_CRYPTO_LIB_SHA256
+		{ 0xba, 0x78, 0x16, 0xbf, 0x8f, 0x01, 0xcf, 0xea,
+		  0x41, 0x41, 0x40, 0xde, 0x5d, 0xae, 0x22, 0x23,
+		  0xb0, 0x03, 0x61, 0xa3, 0x96, 0x17, 0x7a, 0x9c,
+		  0xb4, 0x10, 0xff, 0x61, 0xf2, 0x00, 0x15, 0xad };
+#else /* CONFIG_CRYPTO_LIB_SHA256 */
+		{ 0xa9, 0x99, 0x3e, 0x36, 0x47, 0x06, 0x81, 0x6a, 0xba, 0x3e,
+		  0x25, 0x71, 0x78, 0x50, 0xc2, 0x6c, 0x9c, 0xd0, 0xd8, 0x9d };
+#endif /* CONFIG_CRYPTO_LIB_SHA256 */
+	static const u8 hash_input[] = { 0x61, 0x62, 0x63 }; /* "abc" */
+	u8 digest[sizeof(lrng_hash_selftest_result)] __aligned(sizeof(u32));
+
+	BUG_ON(sizeof(digest) != crypto_cb->lrng_hash_digestsize(NULL));
+
+	if (!crypto_cb->lrng_hash_init(shash, NULL) &&
+	    !crypto_cb->lrng_hash_update(shash, hash_input,
+					 sizeof(hash_input)) &&
+	    !crypto_cb->lrng_hash_final(shash, digest) &&
+	    !memcmp(digest, lrng_hash_selftest_result, sizeof(digest)))
+		return 0;
+
+	pr_err("LRNG %s Hash self-test FAILED\n", crypto_cb->lrng_hash_name());
+	return LRNG_SEFLTEST_ERROR_HASH;
+}
+
+/*
+ * The test vectors were generated using the ChaCha20 DRNG from
+ * https://www.chronox.de/chacha20.html
+ */
+static unsigned int lrng_chacha20_drng_selftest(void)
+{
+	const struct lrng_crypto_cb *crypto_cb = &lrng_cc20_crypto_cb;
+	u8 seed[CHACHA_KEY_SIZE * 2] = {
+		0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+		0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+		0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
+		0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
+		0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27,
+		0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f,
+		0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37,
+		0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f,
+	};
+	struct chacha20_block chacha20;
+	int ret;
+	u8 outbuf[CHACHA_KEY_SIZE * 2] __aligned(sizeof(u32));
+
+	/*
+	 * Expected result when ChaCha20 DRNG state is zero:
+	 *	* constants are set to "expand 32-byte k"
+	 *	* remaining state is 0
+	 * and pulling one half ChaCha20 DRNG block.
+	 */
+	static const u8 expected_halfblock[CHACHA_KEY_SIZE] = {
+		0x76, 0xb8, 0xe0, 0xad, 0xa0, 0xf1, 0x3d, 0x90,
+		0x40, 0x5d, 0x6a, 0xe5, 0x53, 0x86, 0xbd, 0x28,
+		0xbd, 0xd2, 0x19, 0xb8, 0xa0, 0x8d, 0xed, 0x1a,
+		0xa8, 0x36, 0xef, 0xcc, 0x8b, 0x77, 0x0d, 0xc7 };
+
+	/*
+	 * Expected result when ChaCha20 DRNG state is zero:
+	 *	* constants are set to "expand 32-byte k"
+	 *	* remaining state is 0
+	 * followed by a reseed with two keyblocks
+	 *	0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+	 *	0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+	 *	0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
+	 *	0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
+	 *	0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27,
+	 *	0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f,
+	 *	0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37,
+	 *	0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f
+	 * and pulling one ChaCha20 DRNG block.
+	 */
+	static const u8 expected_oneblock[CHACHA_KEY_SIZE * 2] = {
+		0xf5, 0xb4, 0xb6, 0x5a, 0xec, 0xcd, 0x5a, 0x65,
+		0x87, 0x56, 0xe3, 0x86, 0x51, 0x54, 0xfc, 0x90,
+		0x56, 0xff, 0x5e, 0xae, 0x58, 0xf2, 0x01, 0x88,
+		0xb1, 0x7e, 0xb8, 0x2e, 0x17, 0x9a, 0x27, 0xe6,
+		0x86, 0xb3, 0xed, 0x33, 0xf7, 0xb9, 0x06, 0x05,
+		0x8a, 0x2d, 0x1a, 0x93, 0xc9, 0x0b, 0x80, 0x04,
+		0x03, 0xaa, 0x60, 0xaf, 0xd5, 0x36, 0x40, 0x11,
+		0x67, 0x89, 0xb1, 0x66, 0xd5, 0x88, 0x62, 0x6d };
+
+	/*
+	 * Expected result when ChaCha20 DRNG state is zero:
+	 *	* constants are set to "expand 32-byte k"
+	 *	* remaining state is 0
+	 * followed by a reseed with one key block plus one byte
+	 *	0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+	 *	0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+	 *	0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
+	 *	0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
+	 *	0x20
+	 * and pulling less than one ChaCha20 DRNG block.
+	 */
+	static const u8 expected_block_nonalinged[CHACHA_KEY_SIZE + 4] = {
+		0x9d, 0xdd, 0x4f, 0xbe, 0x97, 0xcd, 0x8e, 0x15,
+		0xb3, 0xc4, 0x1a, 0x17, 0x49, 0x29, 0x32, 0x7c,
+		0xb3, 0x84, 0xa4, 0x9b, 0xa7, 0x14, 0xb3, 0xc1,
+		0x5b, 0x3b, 0xfb, 0xa1, 0xe4, 0x23, 0x42, 0x8e,
+		0x08, 0x1f, 0x53, 0xa2 };
+
+	BUILD_BUG_ON(sizeof(seed) % sizeof(u32));
+
+	memset(&chacha20, 0, sizeof(chacha20));
+	lrng_cc20_init_rfc7539(&chacha20);
+	lrng_selftest_bswap32((u32 *)seed, sizeof(seed) / sizeof(u32));
+
+	/* Generate with zero state */
+	ret = crypto_cb->lrng_drng_generate_helper(&chacha20, outbuf,
+						   sizeof(expected_halfblock));
+	if (ret != sizeof(expected_halfblock))
+		goto err;
+	if (memcmp(outbuf, expected_halfblock, sizeof(expected_halfblock)))
+		goto err;
+
+	/* Clear state of DRNG */
+	memset(&chacha20.key.u[0], 0, 48);
+
+	/* Reseed with 2 key blocks */
+	ret = crypto_cb->lrng_drng_seed_helper(&chacha20, seed,
+					       sizeof(expected_oneblock));
+	if (ret < 0)
+		goto err;
+	ret = crypto_cb->lrng_drng_generate_helper(&chacha20, outbuf,
+						   sizeof(expected_oneblock));
+	if (ret != sizeof(expected_oneblock))
+		goto err;
+	if (memcmp(outbuf, expected_oneblock, sizeof(expected_oneblock)))
+		goto err;
+
+	/* Clear state of DRNG */
+	memset(&chacha20.key.u[0], 0, 48);
+
+	/* Reseed with 1 key block and one byte */
+	ret = crypto_cb->lrng_drng_seed_helper(&chacha20, seed,
+					sizeof(expected_block_nonalinged));
+	if (ret < 0)
+		goto err;
+	ret = crypto_cb->lrng_drng_generate_helper(&chacha20, outbuf,
+					sizeof(expected_block_nonalinged));
+	if (ret != sizeof(expected_block_nonalinged))
+		goto err;
+	if (memcmp(outbuf, expected_block_nonalinged,
+		   sizeof(expected_block_nonalinged)))
+		goto err;
+
+	return LRNG_SELFTEST_PASSED;
+
+err:
+	pr_err("LRNG ChaCha20 DRNG self-test FAILED\n");
+	return LRNG_SEFLTEST_ERROR_CHACHA20;
+}
+
+static int lrng_selftest(void)
+{
+	unsigned int ret = lrng_data_process_selftest();
+
+	ret |= lrng_chacha20_drng_selftest();
+	ret |= lrng_hash_selftest();
+
+	if (ret) {
+		if (IS_ENABLED(CONFIG_LRNG_SELFTEST_PANIC))
+			panic("LRNG self-tests failed: %u\n", ret);
+	} else {
+		pr_info("LRNG self-tests passed\n");
+	}
+
+	lrng_selftest_status = ret;
+
+	if (lrng_selftest_status)
+		return -EFAULT;
+	return 0;
+}
+
+#ifdef CONFIG_SYSFS
+/* Re-perform self-test when any value is written to the sysfs file. */
+static int lrng_selftest_sysfs_set(const char *val,
+				   const struct kernel_param *kp)
+{
+	return lrng_selftest();
+}
+
+static const struct kernel_param_ops lrng_selftest_sysfs = {
+	.set = lrng_selftest_sysfs_set,
+	.get = param_get_uint,
+};
+module_param_cb(selftest_status, &lrng_selftest_sysfs, &lrng_selftest_status,
+		0644);
+#endif	/* CONFIG_SYSFS */
+
+static int __init lrng_selftest_init(void)
+{
+	return lrng_selftest();
+}
+
+module_init(lrng_selftest_init);
-- 
2.26.2

Re: [PATCH v36 00/13] /dev/random - a new approach

[Torsten Duwe]

On Mon, 19 Oct 2020 21:28:50 +0200
Stephan Müller <smueller@chronox.de> wrote:
[...]
> * Sole use of crypto for data processing:
[...]
>  - The LRNG uses only properly defined and implemented cryptographic
>    algorithms unlike the use of the SHA-1 transformation in the
> existing /dev/random implementation.
> 
>  - Hash operations use NUMA-node-local hash instances to benefit large
>    parallel systems.
> 
>  - LRNG uses limited number of data post-processing steps
[...]
> * Performance
> 
>  - Faster by up to 75% in the critical code path of the interrupt
> handler depending on data collection size configurable at kernel
> compile time - the default is about equal in performance with
> existing /dev/random as outlined in [2] section 4.2.

[...]
>  - ChaCha20 DRNG is significantly faster as implemented in the
> existing /dev/random as demonstrated with [2] table 2.
> 
>  - Faster entropy collection during boot time to reach fully seeded
>    level, including on virtual systems or systems with SSDs as
> outlined in [2] section 4.1.
> 
> * Testing
[...]

So we now have 2 proposals for a state-of-the-art RNG, and over a month
without a single comment on-topic from any `get_maintainer.pl`

I don't want to emphasise the certification aspects so much. The
interrelation is rather that those certifications require certain code
features, features which are reasonable per se. But the current code is
lagging way behind.

I see the focus namely on performance, scalability, testability and
virtualisation. And it certainly is an advantage to use the code
already present under crypto, with its optimisations, and not rely
on some home brew.

Can we please have a discussion about how to proceed?
Ted, Greg, Arnd: which approach would you prefer?

	Torsten

Re: [PATCH v36 00/13] /dev/random - a new approach

[Greg Kroah-Hartman]

On Wed, Oct 28, 2020 at 06:51:17PM +0100, Torsten Duwe wrote:
> On Mon, 19 Oct 2020 21:28:50 +0200
> Stephan Müller <smueller@chronox.de> wrote:
> [...]
> > * Sole use of crypto for data processing:
> [...]
> >  - The LRNG uses only properly defined and implemented cryptographic
> >    algorithms unlike the use of the SHA-1 transformation in the
> > existing /dev/random implementation.
> > 
> >  - Hash operations use NUMA-node-local hash instances to benefit large
> >    parallel systems.
> > 
> >  - LRNG uses limited number of data post-processing steps
> [...]
> > * Performance
> > 
> >  - Faster by up to 75% in the critical code path of the interrupt
> > handler depending on data collection size configurable at kernel
> > compile time - the default is about equal in performance with
> > existing /dev/random as outlined in [2] section 4.2.
> 
> [...]
> >  - ChaCha20 DRNG is significantly faster as implemented in the
> > existing /dev/random as demonstrated with [2] table 2.
> > 
> >  - Faster entropy collection during boot time to reach fully seeded
> >    level, including on virtual systems or systems with SSDs as
> > outlined in [2] section 4.1.
> > 
> > * Testing
> [...]
> 
> So we now have 2 proposals for a state-of-the-art RNG, and over a month
> without a single comment on-topic from any `get_maintainer.pl`
> 
> I don't want to emphasise the certification aspects so much. The
> interrelation is rather that those certifications require certain code
> features, features which are reasonable per se. But the current code is
> lagging way behind.
> 
> I see the focus namely on performance, scalability, testability and
> virtualisation. And it certainly is an advantage to use the code
> already present under crypto, with its optimisations, and not rely
> on some home brew.
> 
> Can we please have a discussion about how to proceed?
> Ted, Greg, Arnd: which approach would you prefer?

Greg and Arnd are not the random driver maintainers, as is now correctly
shown in the 5.10-rc1 MAINTAINERS file, so I doubt we (well at least I)
have any say here, sorry.

good luck!

greg k-h

Re: [PATCH v36 00/13] /dev/random - a new approach

[Torsten Duwe]

On Wed, 28 Oct 2020 19:07:28 +0100
Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:

> On Wed, Oct 28, 2020 at 06:51:17PM +0100, Torsten Duwe wrote:
> > On Mon, 19 Oct 2020 21:28:50 +0200
> > Stephan Müller <smueller@chronox.de> wrote:
> > [...]
> > > * Sole use of crypto for data processing:
> > [...]
> > >  - The LRNG uses only properly defined and implemented
> > > cryptographic algorithms unlike the use of the SHA-1
> > > transformation in the existing /dev/random implementation.
> > > 
> > >  - Hash operations use NUMA-node-local hash instances to benefit
> > > large parallel systems.
> > > 
> > >  - LRNG uses limited number of data post-processing steps
> > [...]
> > > * Performance
> > > 
> > >  - Faster by up to 75% in the critical code path of the interrupt
> > > handler depending on data collection size configurable at kernel
> > > compile time - the default is about equal in performance with
> > > existing /dev/random as outlined in [2] section 4.2.
> > 
> > [...]
> > >  - ChaCha20 DRNG is significantly faster as implemented in the
> > > existing /dev/random as demonstrated with [2] table 2.
> > > 
> > >  - Faster entropy collection during boot time to reach fully
> > > seeded level, including on virtual systems or systems with SSDs as
> > > outlined in [2] section 4.1.
> > > 
> > > * Testing
> > [...]
> > 
> > So we now have 2 proposals for a state-of-the-art RNG, and over a
> > month without a single comment on-topic from any `get_maintainer.pl`
> > 
> > I don't want to emphasise the certification aspects so much. The
> > interrelation is rather that those certifications require certain
> > code features, features which are reasonable per se. But the
> > current code is lagging way behind.
> > 
> > I see the focus namely on performance, scalability, testability and
> > virtualisation. And it certainly is an advantage to use the code
> > already present under crypto, with its optimisations, and not rely
> > on some home brew.
> > 
> > Can we please have a discussion about how to proceed?
> > Ted, Greg, Arnd: which approach would you prefer?
> 
> Greg and Arnd are not the random driver maintainers, as is now
> correctly shown in the 5.10-rc1 MAINTAINERS file, so I doubt we (well
> at least I) have any say here, sorry.

No problem. get_maintainer (for the proposals) works on paths, not on
topics and I didn't want to leave anybody out.

Ted, if you don't have the time any more to take care of /dev/random,
it's not a shame to hand over maintainership, especially given your
long history of Linux contributions.

Please do seriously consider to hand it over to someone new. This would
be a good opportunity.

	Torsten

Re: [PATCH v36 00/13] /dev/random - a new approach

[Marcelo Henrique Cerri]

[-- Attachment #1: Type: text/plain, Size: 3352 bytes --]

On Mon, Nov 02, 2020 at 02:44:35PM +0100, Torsten Duwe wrote:
> On Wed, 28 Oct 2020 19:07:28 +0100
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> 
> > On Wed, Oct 28, 2020 at 06:51:17PM +0100, Torsten Duwe wrote:
> > > On Mon, 19 Oct 2020 21:28:50 +0200
> > > Stephan Müller <smueller@chronox.de> wrote:
> > > [...]
> > > > * Sole use of crypto for data processing:
> > > [...]
> > > >  - The LRNG uses only properly defined and implemented
> > > > cryptographic algorithms unlike the use of the SHA-1
> > > > transformation in the existing /dev/random implementation.
> > > > 
> > > >  - Hash operations use NUMA-node-local hash instances to benefit
> > > > large parallel systems.
> > > > 
> > > >  - LRNG uses limited number of data post-processing steps
> > > [...]
> > > > * Performance
> > > > 
> > > >  - Faster by up to 75% in the critical code path of the interrupt
> > > > handler depending on data collection size configurable at kernel
> > > > compile time - the default is about equal in performance with
> > > > existing /dev/random as outlined in [2] section 4.2.
> > > 
> > > [...]
> > > >  - ChaCha20 DRNG is significantly faster as implemented in the
> > > > existing /dev/random as demonstrated with [2] table 2.
> > > > 
> > > >  - Faster entropy collection during boot time to reach fully
> > > > seeded level, including on virtual systems or systems with SSDs as
> > > > outlined in [2] section 4.1.
> > > > 
> > > > * Testing
> > > [...]
> > > 
> > > So we now have 2 proposals for a state-of-the-art RNG, and over a
> > > month without a single comment on-topic from any `get_maintainer.pl`
> > > 
> > > I don't want to emphasise the certification aspects so much. The
> > > interrelation is rather that those certifications require certain
> > > code features, features which are reasonable per se. But the
> > > current code is lagging way behind.
> > > 
> > > I see the focus namely on performance, scalability, testability and
> > > virtualisation. And it certainly is an advantage to use the code
> > > already present under crypto, with its optimisations, and not rely
> > > on some home brew.
> > > 
> > > Can we please have a discussion about how to proceed?
> > > Ted, Greg, Arnd: which approach would you prefer?
> > 
> > Greg and Arnd are not the random driver maintainers, as is now
> > correctly shown in the 5.10-rc1 MAINTAINERS file, so I doubt we (well
> > at least I) have any say here, sorry.
> 
> No problem. get_maintainer (for the proposals) works on paths, not on
> topics and I didn't want to leave anybody out.
> 
> Ted, if you don't have the time any more to take care of /dev/random,
> it's not a shame to hand over maintainership, especially given your
> long history of Linux contributions.
> 
> Please do seriously consider to hand it over to someone new. This would
> be a good opportunity.
> 
> 	Torsten
>

I'd like to help with any solution upstream decide to follow either
testing or with code. I understand some of the concerns the community
has regarding FIPS but that doesn't make it less relevant and it's
totally possible to improve /dev/random while allowing it users to
decide if they want to comply to SP 800 90B. I believe the main
blocker now is the lack of direction.

-- 
Regards,
Marcelo


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH v36 00/13] /dev/random - a new approach

[Stephan Mueller]

Am Montag, 19. Oktober 2020, 21:28:50 CET schrieb Stephan Müller:

Hi,
> 
> * Performance
> 
>  - Faster by up to 75% in the critical code path of the interrupt handler
>    depending on data collection size configurable at kernel compile time -
>    the default is about equal in performance with existing /dev/random as
>    outlined in [2] section 4.2.

By streamlining the implementation a bit, the LRNG interrupt handler now 
operates about 130% faster than the existing /dev/random (average of 97 cycles 
of the existing /dev/random code vs. an average of 42 cycles of the LRNG). 
This fast operation is the default now due to patch [2]. The conceptual data 
handling outlined in [3] section 2.2 remains unchanged.

Even the addition of health tests applied to the noise source data would still 
result in a faster interrupt handling code (average of 97 cycles of the 
existing /dev/random code vs on average 78 cycles of the LRNG).

[1] https://github.com/smuellerDD/lrng/commit/
10b74b242950371273e38df78060e258d9d3ea40

[2] https://github.com/smuellerDD/lrng/commit/
383b087653c21cf20984f5508befa57e96f685ba

[3] https://chronox.de/lrng/doc/lrng.pdf

Ciao
Stephan

Re: [PATCH v36 00/13] /dev/random - a new approach

[Torsten Duwe]

On Mon, Nov 02, 2020 at 02:44:35PM +0100, Torsten Duwe wrote:
> 
> Ted, if you don't have the time any more to take care of /dev/random,
> it's not a shame to hand over maintainership, especially given your
> long history of Linux contributions.
> 
> Please do seriously consider to hand it over to someone new. This would
> be a good opportunity.

I can see you are quite busy working on ext4, and there is a number of
patches for drivers/char/random.c awaiting review. Wouldn't it be good
to pass it on to someone more enthusiastic?

At least some sort of reply would be appreciated.
Or are you already pondering the request ;-) ?

	Torsten