* [PATCH] random: handle archrandom in plural words @ 2022-07-17 11:24 Jason A. Donenfeld 2022-07-17 11:43 ` Jason A. Donenfeld 0 siblings, 1 reply; 27+ messages in thread From: Jason A. Donenfeld @ 2022-07-17 11:24 UTC (permalink / raw) To: linux-kernel Cc: Jason A. Donenfeld, Will Deacon, Michael Ellerman, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Harald Freudenberger The archrandom interface was originally designed for x86, which supplies RDRAND/RDSEED for receiving random words into registers, resulting in one function to generate an int and another to generate a long. However, other architectures don't follow this. On arm64, the SMCCC TRNG interface can return between 1 and 3 words. On s390, the CPACF TRNG interface can return between 1 and 32 words for the same cost as for one word. On UML, the os_getrandom() interface can return arbitrary amounts. So change the api signature to take a "words" parameter designating the maximum number of words requested, and then return the number of words generated. Since callers need to check this return value and loop anyway, each arch implementation does not bother implementing its own loop to try again to fill the requested number of words. Additionally, all existing callers pass in a constant words parameter. Taken together, these two things mean that the codegen doesn't really change much for one-word-at-a-time platforms, while performance is greatly improved on platforms such as s390. Cc: Will Deacon <will@kernel.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Borislav Petkov <bp@suse.de> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Johannes Berg <johannes@sipsolutions.net> Cc: Harald Freudenberger <freude@linux.ibm.com> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> --- arch/arm64/include/asm/archrandom.h | 102 ++++++++++++-------------- arch/arm64/kernel/kaslr.c | 2 +- arch/powerpc/include/asm/archrandom.h | 30 ++------ arch/powerpc/kvm/book3s_hv.c | 2 +- arch/s390/include/asm/archrandom.h | 25 ++----- arch/um/include/asm/archrandom.h | 21 ++---- arch/x86/include/asm/archrandom.h | 41 +---------- arch/x86/kernel/espfix_64.c | 2 +- drivers/char/random.c | 45 ++++++++---- include/asm-generic/archrandom.h | 14 +--- include/linux/random.h | 12 +-- 11 files changed, 111 insertions(+), 185 deletions(-) diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h index c3b9fa56af67..7a24fdee3e2f 100644 --- a/arch/arm64/include/asm/archrandom.h +++ b/arch/arm64/include/asm/archrandom.h @@ -58,7 +58,7 @@ static inline bool __arm64_rndrrs(unsigned long *v) return ok; } -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { /* * Only support the generic interface after we have detected @@ -66,27 +66,15 @@ static inline bool __must_check arch_get_random_long(unsigned long *v) * cpufeature code and with potential scheduling between CPUs * with and without the feature. */ - if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) - return true; - return false; + if (words && cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) + return 1; + return 0; } -static inline bool __must_check arch_get_random_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { - if (cpus_have_const_cap(ARM64_HAS_RNG)) { - unsigned long val; - - if (__arm64_rndr(&val)) { - *v = val; - return true; - } - } - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - struct arm_smccc_res res; + if (!words) + return 0; /* * We prefer the SMCCC call, since its semantics (return actual @@ -95,10 +83,23 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) * (the output of a pseudo RNG freshly seeded by a TRNG). */ if (smccc_trng_available) { - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); + struct arm_smccc_res res; + + words = min_t(size_t, 3, words); + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, words * 64, &res); if ((int)res.a0 >= 0) { - *v = res.a3; - return true; + switch (words) { + case 3: + *v++ = res.a1; + fallthrough; + case 2: + *v++ = res.a2; + fallthrough; + case 1: + *v++ = res.a3; + break; + } + return words; } } @@ -108,32 +109,9 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) * enough to implement this API if no other entropy source exists. */ if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndrrs(v)) - return true; + return 1; - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) -{ - struct arm_smccc_res res; - unsigned long val; - - if (smccc_trng_available) { - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 32, &res); - if ((int)res.a0 >= 0) { - *v = res.a3 & GENMASK(31, 0); - return true; - } - } - - if (cpus_have_const_cap(ARM64_HAS_RNG)) { - if (__arm64_rndrrs(&val)) { - *v = val; - return true; - } - } - - return false; + return 0; } static inline bool __init __early_cpu_has_rndr(void) @@ -143,26 +121,40 @@ static inline bool __init __early_cpu_has_rndr(void) return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; } -static inline bool __init __must_check -arch_get_random_seed_long_early(unsigned long *v) +static inline size_t __init __must_check +arch_get_random_seed_words_early(unsigned long *v, size_t words) { WARN_ON(system_state != SYSTEM_BOOTING); + if (!words) + return 0; + if (smccc_trng_available) { struct arm_smccc_res res; - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); + words = min_t(size_t, 3, words); + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, words * 64, &res); if ((int)res.a0 >= 0) { - *v = res.a3; - return true; + switch (words) { + case 3: + *v++ = res.a1; + fallthrough; + case 2: + *v++ = res.a2; + fallthrough; + case 1: + *v++ = res.a3; + break; + } + return words; } } if (__early_cpu_has_rndr() && __arm64_rndr(v)) - return true; + return 1; - return false; + return 0; } -#define arch_get_random_seed_long_early arch_get_random_seed_long_early +#define arch_get_random_seed_words_early arch_get_random_seed_words_early #endif /* _ASM_ARCHRANDOM_H */ diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c index 418b2bba1521..ed77afe16121 100644 --- a/arch/arm64/kernel/kaslr.c +++ b/arch/arm64/kernel/kaslr.c @@ -106,7 +106,7 @@ u64 __init kaslr_early_init(void) * and supported. */ - if (arch_get_random_seed_long_early(&raw)) + if (arch_get_random_seed_words_early(&raw, 1)) seed ^= raw; if (!seed) { diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h index 25ba65df6b1a..bf2182f80480 100644 --- a/arch/powerpc/include/asm/archrandom.h +++ b/arch/powerpc/include/asm/archrandom.h @@ -4,34 +4,16 @@ #include <asm/machdep.h> -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { - return false; + return 0; } -static inline bool __must_check arch_get_random_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - if (ppc_md.get_random_seed) - return ppc_md.get_random_seed(v); - - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) -{ - unsigned long val; - bool rc; - - rc = arch_get_random_seed_long(&val); - if (rc) - *v = val; - - return rc; + if (words && ppc_md.get_random_seed && ppc_md.get_random_seed(v)) + return 1; + return 0; } #ifdef CONFIG_PPC_POWERNV diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index e08fb3124dca..18b2d80996b6 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -1207,7 +1207,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) break; #endif case H_RANDOM: - if (!arch_get_random_seed_long(&vcpu->arch.regs.gpr[4])) + if (!arch_get_random_seed_words(&vcpu->arch.regs.gpr[4], 1)) ret = H_HARDWARE; break; case H_RPT_INVALIDATE: diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h index 0a1c2e66c709..5ada507a1fb8 100644 --- a/arch/s390/include/asm/archrandom.h +++ b/arch/s390/include/asm/archrandom.h @@ -18,31 +18,16 @@ DECLARE_STATIC_KEY_FALSE(s390_arch_random_available); extern atomic64_t s390_arch_random_counter; -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { - return false; -} - -static inline bool __must_check arch_get_random_int(unsigned int *v) -{ - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - if (static_branch_likely(&s390_arch_random_available)) { - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); - atomic64_add(sizeof(*v), &s390_arch_random_counter); - return true; - } - return false; + return 0; } -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { if (static_branch_likely(&s390_arch_random_available)) { - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); - atomic64_add(sizeof(*v), &s390_arch_random_counter); + cpacf_trng(NULL, 0, (u8 *)v, words * sizeof(*v)); + atomic64_add(words * sizeof(*v), &s390_arch_random_counter); return true; } return false; diff --git a/arch/um/include/asm/archrandom.h b/arch/um/include/asm/archrandom.h index 2f24cb96391d..1dd9207c42f8 100644 --- a/arch/um/include/asm/archrandom.h +++ b/arch/um/include/asm/archrandom.h @@ -7,24 +7,17 @@ /* This is from <os.h>, but better not to #include that in a global header here. */ ssize_t os_getrandom(void *buf, size_t len, unsigned int flags); -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); + ssize_t ret = os_getrandom(v, words * sizeof(*v), 0); + if (ret < 0) + return 0; + return ret; } -static inline bool __must_check arch_get_random_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) -{ - return false; + return 0; } #endif diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h index fb235b696175..a1717b81d876 100644 --- a/arch/x86/include/asm/archrandom.h +++ b/arch/x86/include/asm/archrandom.h @@ -31,20 +31,6 @@ static inline bool __must_check rdrand_long(unsigned long *v) return false; } -static inline bool __must_check rdrand_int(unsigned int *v) -{ - bool ok; - unsigned int retry = RDRAND_RETRY_LOOPS; - do { - asm volatile("rdrand %[out]" - CC_SET(c) - : CC_OUT(c) (ok), [out] "=r" (*v)); - if (ok) - return true; - } while (--retry); - return false; -} - static inline bool __must_check rdseed_long(unsigned long *v) { bool ok; @@ -54,38 +40,19 @@ static inline bool __must_check rdseed_long(unsigned long *v) return ok; } -static inline bool __must_check rdseed_int(unsigned int *v) -{ - bool ok; - asm volatile("rdseed %[out]" - CC_SET(c) - : CC_OUT(c) (ok), [out] "=r" (*v)); - return ok; -} - /* * These are the generic interfaces; they must not be declared if the * stubs in <linux/random.h> are to be invoked. */ -static inline bool __must_check arch_get_random_long(unsigned long *v) -{ - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_long(v) : false; -} - -static inline bool __must_check arch_get_random_int(unsigned int *v) -{ - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_int(v) : false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_long(v) : false; + return words && static_cpu_has(X86_FEATURE_RDRAND) && rdrand_long(v) ? 1 : 0; } -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_int(v) : false; + return words && static_cpu_has(X86_FEATURE_RDSEED) && rdseed_long(v) ? 1 : 0; } #ifndef CONFIG_UML diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c index 4fe7af58cfe1..f46c9ff3c0d4 100644 --- a/arch/x86/kernel/espfix_64.c +++ b/arch/x86/kernel/espfix_64.c @@ -100,7 +100,7 @@ static void init_espfix_random(void) * This is run before the entropy pools are initialized, * but this is hopefully better than nothing. */ - if (!arch_get_random_long(&rand)) { + if (!arch_get_random_words(&rand, 1)) { /* The constant is an arbitrary large prime */ rand = rdtsc(); rand *= 0xc345c6b72fd16123UL; diff --git a/drivers/char/random.c b/drivers/char/random.c index 0c6568ae5f68..70d8d1d7e2d7 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -596,12 +596,20 @@ static void extract_entropy(void *buf, size_t len) unsigned long rdseed[32 / sizeof(long)]; size_t counter; } block; - size_t i; + size_t i, words; - for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) { - if (!arch_get_random_seed_long(&block.rdseed[i]) && - !arch_get_random_long(&block.rdseed[i])) - block.rdseed[i] = random_get_entropy(); + for (i = 0; i < ARRAY_SIZE(block.rdseed);) { + words = arch_get_random_seed_words(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); + if (words) { + i += words; + continue; + } + words = arch_get_random_words(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); + if (words) { + i += words; + continue; + } + block.rdseed[i++] = random_get_entropy(); } spin_lock_irqsave(&input_pool.lock, flags); @@ -776,22 +784,31 @@ static struct notifier_block pm_notifier = { .notifier_call = random_pm_notifica int __init random_init(const char *command_line) { ktime_t now = ktime_get_real(); - unsigned int i, arch_bits; - unsigned long entropy; + size_t i, words, arch_bits; + unsigned long entropy[BLAKE2S_BLOCK_SIZE / sizeof(long)]; #if defined(LATENT_ENTROPY_PLUGIN) static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst __latent_entropy; _mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed)); #endif - for (i = 0, arch_bits = BLAKE2S_BLOCK_SIZE * 8; - i < BLAKE2S_BLOCK_SIZE; i += sizeof(entropy)) { - if (!arch_get_random_seed_long_early(&entropy) && - !arch_get_random_long_early(&entropy)) { - entropy = random_get_entropy(); - arch_bits -= sizeof(entropy) * 8; + for (i = 0, arch_bits = sizeof(entropy) * 8; i < ARRAY_SIZE(entropy);) { + words = arch_get_random_seed_words(entropy, ARRAY_SIZE(entropy) - i); + if (words) { + _mix_pool_bytes(entropy, sizeof(*entropy) * words); + i += words; + continue; } - _mix_pool_bytes(&entropy, sizeof(entropy)); + words = arch_get_random_words(entropy, ARRAY_SIZE(entropy) - i); + if (words) { + _mix_pool_bytes(entropy, sizeof(*entropy) * words); + i += words; + continue; + } + entropy[0] = random_get_entropy(); + _mix_pool_bytes(entropy, sizeof(*entropy)); + arch_bits -= sizeof(*entropy) * 8; + ++i; } _mix_pool_bytes(&now, sizeof(now)); _mix_pool_bytes(utsname(), sizeof(*(utsname()))); diff --git a/include/asm-generic/archrandom.h b/include/asm-generic/archrandom.h index 3a5ee202dd86..4cdf9ab2b6cc 100644 --- a/include/asm-generic/archrandom.h +++ b/include/asm-generic/archrandom.h @@ -2,22 +2,12 @@ #ifndef __ASM_GENERIC_ARCHRANDOM_H__ #define __ASM_GENERIC_ARCHRANDOM_H__ -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { return false; } -static inline bool __must_check arch_get_random_int(unsigned int *v) -{ - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { return false; } diff --git a/include/linux/random.h b/include/linux/random.h index 865770e29f3e..0a327a289f09 100644 --- a/include/linux/random.h +++ b/include/linux/random.h @@ -112,19 +112,19 @@ declare_get_random_var_wait(long, unsigned long) * Called from the boot CPU during startup; not valid to call once * secondary CPUs are up and preemption is possible. */ -#ifndef arch_get_random_seed_long_early -static inline bool __init arch_get_random_seed_long_early(unsigned long *v) +#ifndef arch_get_random_seed_words_early +static inline size_t __init arch_get_random_seed_words_early(unsigned long *v, size_t words) { WARN_ON(system_state != SYSTEM_BOOTING); - return arch_get_random_seed_long(v); + return arch_get_random_seed_words(v, words); } #endif -#ifndef arch_get_random_long_early -static inline bool __init arch_get_random_long_early(unsigned long *v) +#ifndef arch_get_random_words_early +static inline bool __init arch_get_random_words_early(unsigned long *v, size_t words) { WARN_ON(system_state != SYSTEM_BOOTING); - return arch_get_random_long(v); + return arch_get_random_words(v, words); } #endif -- 2.35.1 ^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH] random: handle archrandom in plural words 2022-07-17 11:24 [PATCH] random: handle archrandom in plural words Jason A. Donenfeld @ 2022-07-17 11:43 ` Jason A. Donenfeld 2022-07-17 20:03 ` [PATCH v2] " Jason A. Donenfeld 0 siblings, 1 reply; 27+ messages in thread From: Jason A. Donenfeld @ 2022-07-17 11:43 UTC (permalink / raw) To: linux-kernel Cc: Will Deacon, Michael Ellerman, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Harald Freudenberger On Sun, Jul 17, 2022 at 01:24:39PM +0200, Jason A. Donenfeld wrote: > The archrandom interface was originally designed for x86, which supplies > RDRAND/RDSEED for receiving random words into registers, resulting in > one function to generate an int and another to generate a long. However, > other architectures don't follow this. > > On arm64, the SMCCC TRNG interface can return between 1 and 3 words. On > s390, the CPACF TRNG interface can return between 1 and 32 words for the > same cost as for one word. On UML, the os_getrandom() interface can return > arbitrary amounts. > > So change the api signature to take a "words" parameter designating the > maximum number of words requested, and then return the number of words > generated. > > Since callers need to check this return value and loop anyway, each arch > implementation does not bother implementing its own loop to try again to > fill the requested number of words. Additionally, all existing callers > pass in a constant words parameter. Taken together, these two things > mean that the codegen doesn't really change much for one-word-at-a-time > platforms, while performance is greatly improved on platforms such as > s390. > > Cc: Will Deacon <will@kernel.org> > Cc: Michael Ellerman <mpe@ellerman.id.au> > Cc: Alexander Gordeev <agordeev@linux.ibm.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: H. Peter Anvin <hpa@zytor.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Johannes Berg <johannes@sipsolutions.net> > Cc: Harald Freudenberger <freude@linux.ibm.com> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> > --- > arch/arm64/include/asm/archrandom.h | 102 ++++++++++++-------------- > arch/arm64/kernel/kaslr.c | 2 +- > arch/powerpc/include/asm/archrandom.h | 30 ++------ > arch/powerpc/kvm/book3s_hv.c | 2 +- > arch/s390/include/asm/archrandom.h | 25 ++----- > arch/um/include/asm/archrandom.h | 21 ++---- > arch/x86/include/asm/archrandom.h | 41 +---------- > arch/x86/kernel/espfix_64.c | 2 +- > drivers/char/random.c | 45 ++++++++---- > include/asm-generic/archrandom.h | 14 +--- > include/linux/random.h | 12 +-- > 11 files changed, 111 insertions(+), 185 deletions(-) > > diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h > index c3b9fa56af67..7a24fdee3e2f 100644 > --- a/arch/arm64/include/asm/archrandom.h > +++ b/arch/arm64/include/asm/archrandom.h > @@ -58,7 +58,7 @@ static inline bool __arm64_rndrrs(unsigned long *v) > return ok; > } > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > /* > * Only support the generic interface after we have detected > @@ -66,27 +66,15 @@ static inline bool __must_check arch_get_random_long(unsigned long *v) > * cpufeature code and with potential scheduling between CPUs > * with and without the feature. > */ > - if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) > - return true; > - return false; > + if (words && cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) > + return 1; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - if (cpus_have_const_cap(ARM64_HAS_RNG)) { > - unsigned long val; > - > - if (__arm64_rndr(&val)) { > - *v = val; > - return true; > - } > - } > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - struct arm_smccc_res res; > + if (!words) > + return 0; > > /* > * We prefer the SMCCC call, since its semantics (return actual > @@ -95,10 +83,23 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > * (the output of a pseudo RNG freshly seeded by a TRNG). > */ > if (smccc_trng_available) { > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); > + struct arm_smccc_res res; > + > + words = min_t(size_t, 3, words); > + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, words * 64, &res); > if ((int)res.a0 >= 0) { > - *v = res.a3; > - return true; > + switch (words) { > + case 3: > + *v++ = res.a1; > + fallthrough; > + case 2: > + *v++ = res.a2; > + fallthrough; > + case 1: > + *v++ = res.a3; > + break; > + } > + return words; > } > } > > @@ -108,32 +109,9 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > * enough to implement this API if no other entropy source exists. > */ > if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndrrs(v)) > - return true; > + return 1; > > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - struct arm_smccc_res res; > - unsigned long val; > - > - if (smccc_trng_available) { > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 32, &res); > - if ((int)res.a0 >= 0) { > - *v = res.a3 & GENMASK(31, 0); > - return true; > - } > - } > - > - if (cpus_have_const_cap(ARM64_HAS_RNG)) { > - if (__arm64_rndrrs(&val)) { > - *v = val; > - return true; > - } > - } > - > - return false; > + return 0; > } > > static inline bool __init __early_cpu_has_rndr(void) > @@ -143,26 +121,40 @@ static inline bool __init __early_cpu_has_rndr(void) > return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; > } > > -static inline bool __init __must_check > -arch_get_random_seed_long_early(unsigned long *v) > +static inline size_t __init __must_check > +arch_get_random_seed_words_early(unsigned long *v, size_t words) > { > WARN_ON(system_state != SYSTEM_BOOTING); > > + if (!words) > + return 0; > + > if (smccc_trng_available) { > struct arm_smccc_res res; > > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); > + words = min_t(size_t, 3, words); > + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, words * 64, &res); > if ((int)res.a0 >= 0) { > - *v = res.a3; > - return true; > + switch (words) { > + case 3: > + *v++ = res.a1; > + fallthrough; > + case 2: > + *v++ = res.a2; > + fallthrough; > + case 1: > + *v++ = res.a3; > + break; > + } > + return words; > } > } > > if (__early_cpu_has_rndr() && __arm64_rndr(v)) > - return true; > + return 1; > > - return false; > + return 0; > } > -#define arch_get_random_seed_long_early arch_get_random_seed_long_early > +#define arch_get_random_seed_words_early arch_get_random_seed_words_early > > #endif /* _ASM_ARCHRANDOM_H */ > diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c > index 418b2bba1521..ed77afe16121 100644 > --- a/arch/arm64/kernel/kaslr.c > +++ b/arch/arm64/kernel/kaslr.c > @@ -106,7 +106,7 @@ u64 __init kaslr_early_init(void) > * and supported. > */ > > - if (arch_get_random_seed_long_early(&raw)) > + if (arch_get_random_seed_words_early(&raw, 1)) > seed ^= raw; > > if (!seed) { > diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h > index 25ba65df6b1a..bf2182f80480 100644 > --- a/arch/powerpc/include/asm/archrandom.h > +++ b/arch/powerpc/include/asm/archrandom.h > @@ -4,34 +4,16 @@ > > #include <asm/machdep.h> > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - if (ppc_md.get_random_seed) > - return ppc_md.get_random_seed(v); > - > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - unsigned long val; > - bool rc; > - > - rc = arch_get_random_seed_long(&val); > - if (rc) > - *v = val; > - > - return rc; > + if (words && ppc_md.get_random_seed && ppc_md.get_random_seed(v)) > + return 1; > + return 0; > } > > #ifdef CONFIG_PPC_POWERNV > diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c > index e08fb3124dca..18b2d80996b6 100644 > --- a/arch/powerpc/kvm/book3s_hv.c > +++ b/arch/powerpc/kvm/book3s_hv.c > @@ -1207,7 +1207,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) > break; > #endif > case H_RANDOM: > - if (!arch_get_random_seed_long(&vcpu->arch.regs.gpr[4])) > + if (!arch_get_random_seed_words(&vcpu->arch.regs.gpr[4], 1)) > ret = H_HARDWARE; > break; > case H_RPT_INVALIDATE: > diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h > index 0a1c2e66c709..5ada507a1fb8 100644 > --- a/arch/s390/include/asm/archrandom.h > +++ b/arch/s390/include/asm/archrandom.h > @@ -18,31 +18,16 @@ > DECLARE_STATIC_KEY_FALSE(s390_arch_random_available); > extern atomic64_t s390_arch_random_counter; > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return false; > -} > - > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - if (static_branch_likely(&s390_arch_random_available)) { > - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); > - atomic64_add(sizeof(*v), &s390_arch_random_counter); > - return true; > - } > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > if (static_branch_likely(&s390_arch_random_available)) { > - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); > - atomic64_add(sizeof(*v), &s390_arch_random_counter); > + cpacf_trng(NULL, 0, (u8 *)v, words * sizeof(*v)); > + atomic64_add(words * sizeof(*v), &s390_arch_random_counter); > return true; return words; > } > return false; return 0; > diff --git a/arch/um/include/asm/archrandom.h b/arch/um/include/asm/archrandom.h > index 2f24cb96391d..1dd9207c42f8 100644 > --- a/arch/um/include/asm/archrandom.h > +++ b/arch/um/include/asm/archrandom.h > @@ -7,24 +7,17 @@ > /* This is from <os.h>, but better not to #include that in a global header here. */ > ssize_t os_getrandom(void *buf, size_t len, unsigned int flags); > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); > + ssize_t ret = os_getrandom(v, words * sizeof(*v), 0); > + if (ret < 0) > + return 0; > + return ret; return ret / sizeof(*v); > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - return false; > + return 0; > } > > #endif > diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h > index fb235b696175..a1717b81d876 100644 > --- a/arch/x86/include/asm/archrandom.h > +++ b/arch/x86/include/asm/archrandom.h > @@ -31,20 +31,6 @@ static inline bool __must_check rdrand_long(unsigned long *v) > return false; > } > > -static inline bool __must_check rdrand_int(unsigned int *v) > -{ > - bool ok; > - unsigned int retry = RDRAND_RETRY_LOOPS; > - do { > - asm volatile("rdrand %[out]" > - CC_SET(c) > - : CC_OUT(c) (ok), [out] "=r" (*v)); > - if (ok) > - return true; > - } while (--retry); > - return false; > -} > - > static inline bool __must_check rdseed_long(unsigned long *v) > { > bool ok; > @@ -54,38 +40,19 @@ static inline bool __must_check rdseed_long(unsigned long *v) > return ok; > } > > -static inline bool __must_check rdseed_int(unsigned int *v) > -{ > - bool ok; > - asm volatile("rdseed %[out]" > - CC_SET(c) > - : CC_OUT(c) (ok), [out] "=r" (*v)); > - return ok; > -} > - > /* > * These are the generic interfaces; they must not be declared if the > * stubs in <linux/random.h> are to be invoked. > */ > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > -{ > - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_long(v) : false; > -} > - > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_int(v) : false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_long(v) : false; > + return words && static_cpu_has(X86_FEATURE_RDRAND) && rdrand_long(v) ? 1 : 0; > } > > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_int(v) : false; > + return words && static_cpu_has(X86_FEATURE_RDSEED) && rdseed_long(v) ? 1 : 0; > } > > #ifndef CONFIG_UML > diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c > index 4fe7af58cfe1..f46c9ff3c0d4 100644 > --- a/arch/x86/kernel/espfix_64.c > +++ b/arch/x86/kernel/espfix_64.c > @@ -100,7 +100,7 @@ static void init_espfix_random(void) > * This is run before the entropy pools are initialized, > * but this is hopefully better than nothing. > */ > - if (!arch_get_random_long(&rand)) { > + if (!arch_get_random_words(&rand, 1)) { > /* The constant is an arbitrary large prime */ > rand = rdtsc(); > rand *= 0xc345c6b72fd16123UL; > diff --git a/drivers/char/random.c b/drivers/char/random.c > index 0c6568ae5f68..70d8d1d7e2d7 100644 > --- a/drivers/char/random.c > +++ b/drivers/char/random.c > @@ -596,12 +596,20 @@ static void extract_entropy(void *buf, size_t len) > unsigned long rdseed[32 / sizeof(long)]; > size_t counter; > } block; > - size_t i; > + size_t i, words; > > - for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) { > - if (!arch_get_random_seed_long(&block.rdseed[i]) && > - !arch_get_random_long(&block.rdseed[i])) > - block.rdseed[i] = random_get_entropy(); > + for (i = 0; i < ARRAY_SIZE(block.rdseed);) { > + words = arch_get_random_seed_words(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); > + if (words) { > + i += words; > + continue; > + } > + words = arch_get_random_words(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); > + if (words) { > + i += words; > + continue; > + } > + block.rdseed[i++] = random_get_entropy(); > } > > spin_lock_irqsave(&input_pool.lock, flags); > @@ -776,22 +784,31 @@ static struct notifier_block pm_notifier = { .notifier_call = random_pm_notifica > int __init random_init(const char *command_line) > { > ktime_t now = ktime_get_real(); > - unsigned int i, arch_bits; > - unsigned long entropy; > + size_t i, words, arch_bits; > + unsigned long entropy[BLAKE2S_BLOCK_SIZE / sizeof(long)]; > > #if defined(LATENT_ENTROPY_PLUGIN) > static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst __latent_entropy; > _mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed)); > #endif > > - for (i = 0, arch_bits = BLAKE2S_BLOCK_SIZE * 8; > - i < BLAKE2S_BLOCK_SIZE; i += sizeof(entropy)) { > - if (!arch_get_random_seed_long_early(&entropy) && > - !arch_get_random_long_early(&entropy)) { > - entropy = random_get_entropy(); > - arch_bits -= sizeof(entropy) * 8; > + for (i = 0, arch_bits = sizeof(entropy) * 8; i < ARRAY_SIZE(entropy);) { > + words = arch_get_random_seed_words(entropy, ARRAY_SIZE(entropy) - i); > + if (words) { > + _mix_pool_bytes(entropy, sizeof(*entropy) * words); > + i += words; > + continue; > } > - _mix_pool_bytes(&entropy, sizeof(entropy)); > + words = arch_get_random_words(entropy, ARRAY_SIZE(entropy) - i); > + if (words) { > + _mix_pool_bytes(entropy, sizeof(*entropy) * words); > + i += words; > + continue; > + } > + entropy[0] = random_get_entropy(); > + _mix_pool_bytes(entropy, sizeof(*entropy)); > + arch_bits -= sizeof(*entropy) * 8; > + ++i; > } > _mix_pool_bytes(&now, sizeof(now)); > _mix_pool_bytes(utsname(), sizeof(*(utsname()))); > diff --git a/include/asm-generic/archrandom.h b/include/asm-generic/archrandom.h > index 3a5ee202dd86..4cdf9ab2b6cc 100644 > --- a/include/asm-generic/archrandom.h > +++ b/include/asm-generic/archrandom.h > @@ -2,22 +2,12 @@ > #ifndef __ASM_GENERIC_ARCHRANDOM_H__ > #define __ASM_GENERIC_ARCHRANDOM_H__ > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > return false; return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > return false; return 0; > } > diff --git a/include/linux/random.h b/include/linux/random.h > index 865770e29f3e..0a327a289f09 100644 > --- a/include/linux/random.h > +++ b/include/linux/random.h > @@ -112,19 +112,19 @@ declare_get_random_var_wait(long, unsigned long) > * Called from the boot CPU during startup; not valid to call once > * secondary CPUs are up and preemption is possible. > */ > -#ifndef arch_get_random_seed_long_early > -static inline bool __init arch_get_random_seed_long_early(unsigned long *v) > +#ifndef arch_get_random_seed_words_early > +static inline size_t __init arch_get_random_seed_words_early(unsigned long *v, size_t words) > { > WARN_ON(system_state != SYSTEM_BOOTING); > - return arch_get_random_seed_long(v); > + return arch_get_random_seed_words(v, words); > } > #endif > > -#ifndef arch_get_random_long_early > -static inline bool __init arch_get_random_long_early(unsigned long *v) > +#ifndef arch_get_random_words_early > +static inline bool __init arch_get_random_words_early(unsigned long *v, size_t words) > { > WARN_ON(system_state != SYSTEM_BOOTING); > - return arch_get_random_long(v); > + return arch_get_random_words(v, words); > } > #endif > > -- > 2.35.1 > ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v2] random: handle archrandom in plural words 2022-07-17 11:43 ` Jason A. Donenfeld @ 2022-07-17 20:03 ` Jason A. Donenfeld 2022-07-18 6:31 ` Michael Ellerman ` (2 more replies) 0 siblings, 3 replies; 27+ messages in thread From: Jason A. Donenfeld @ 2022-07-17 20:03 UTC (permalink / raw) To: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86 Cc: Jason A. Donenfeld, Will Deacon, Michael Ellerman, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Harald Freudenberger The archrandom interface was originally designed for x86, which supplies RDRAND/RDSEED for receiving random words into registers, resulting in one function to generate an int and another to generate a long. However, other architectures don't follow this. On arm64, the SMCCC TRNG interface can return between 1 and 3 words. On s390, the CPACF TRNG interface can return between 1 and 32 words for the same cost as for one word. On UML, the os_getrandom() interface can return arbitrary amounts. So change the api signature to take a "words" parameter designating the maximum number of words requested, and then return the number of words generated. Since callers need to check this return value and loop anyway, each arch implementation does not bother implementing its own loop to try again to fill the requested number of words. Additionally, all existing callers pass in a constant words parameter. Taken together, these two things mean that the codegen doesn't really change much for one-word-at-a-time platforms, while performance is greatly improved on platforms such as s390. Cc: Will Deacon <will@kernel.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Borislav Petkov <bp@suse.de> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Johannes Berg <johannes@sipsolutions.net> Cc: Harald Freudenberger <freude@linux.ibm.com> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> --- arch/arm64/include/asm/archrandom.h | 102 ++++++++++++-------------- arch/arm64/kernel/kaslr.c | 2 +- arch/powerpc/include/asm/archrandom.h | 30 ++------ arch/powerpc/kvm/book3s_hv.c | 2 +- arch/s390/include/asm/archrandom.h | 29 ++------ arch/um/include/asm/archrandom.h | 21 ++---- arch/x86/include/asm/archrandom.h | 41 +---------- arch/x86/kernel/espfix_64.c | 2 +- drivers/char/random.c | 45 ++++++++---- include/asm-generic/archrandom.h | 18 +---- include/linux/random.h | 12 +-- 11 files changed, 116 insertions(+), 188 deletions(-) diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h index c3b9fa56af67..7a24fdee3e2f 100644 --- a/arch/arm64/include/asm/archrandom.h +++ b/arch/arm64/include/asm/archrandom.h @@ -58,7 +58,7 @@ static inline bool __arm64_rndrrs(unsigned long *v) return ok; } -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { /* * Only support the generic interface after we have detected @@ -66,27 +66,15 @@ static inline bool __must_check arch_get_random_long(unsigned long *v) * cpufeature code and with potential scheduling between CPUs * with and without the feature. */ - if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) - return true; - return false; + if (words && cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) + return 1; + return 0; } -static inline bool __must_check arch_get_random_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { - if (cpus_have_const_cap(ARM64_HAS_RNG)) { - unsigned long val; - - if (__arm64_rndr(&val)) { - *v = val; - return true; - } - } - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - struct arm_smccc_res res; + if (!words) + return 0; /* * We prefer the SMCCC call, since its semantics (return actual @@ -95,10 +83,23 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) * (the output of a pseudo RNG freshly seeded by a TRNG). */ if (smccc_trng_available) { - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); + struct arm_smccc_res res; + + words = min_t(size_t, 3, words); + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, words * 64, &res); if ((int)res.a0 >= 0) { - *v = res.a3; - return true; + switch (words) { + case 3: + *v++ = res.a1; + fallthrough; + case 2: + *v++ = res.a2; + fallthrough; + case 1: + *v++ = res.a3; + break; + } + return words; } } @@ -108,32 +109,9 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) * enough to implement this API if no other entropy source exists. */ if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndrrs(v)) - return true; + return 1; - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) -{ - struct arm_smccc_res res; - unsigned long val; - - if (smccc_trng_available) { - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 32, &res); - if ((int)res.a0 >= 0) { - *v = res.a3 & GENMASK(31, 0); - return true; - } - } - - if (cpus_have_const_cap(ARM64_HAS_RNG)) { - if (__arm64_rndrrs(&val)) { - *v = val; - return true; - } - } - - return false; + return 0; } static inline bool __init __early_cpu_has_rndr(void) @@ -143,26 +121,40 @@ static inline bool __init __early_cpu_has_rndr(void) return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; } -static inline bool __init __must_check -arch_get_random_seed_long_early(unsigned long *v) +static inline size_t __init __must_check +arch_get_random_seed_words_early(unsigned long *v, size_t words) { WARN_ON(system_state != SYSTEM_BOOTING); + if (!words) + return 0; + if (smccc_trng_available) { struct arm_smccc_res res; - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); + words = min_t(size_t, 3, words); + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, words * 64, &res); if ((int)res.a0 >= 0) { - *v = res.a3; - return true; + switch (words) { + case 3: + *v++ = res.a1; + fallthrough; + case 2: + *v++ = res.a2; + fallthrough; + case 1: + *v++ = res.a3; + break; + } + return words; } } if (__early_cpu_has_rndr() && __arm64_rndr(v)) - return true; + return 1; - return false; + return 0; } -#define arch_get_random_seed_long_early arch_get_random_seed_long_early +#define arch_get_random_seed_words_early arch_get_random_seed_words_early #endif /* _ASM_ARCHRANDOM_H */ diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c index 418b2bba1521..ed77afe16121 100644 --- a/arch/arm64/kernel/kaslr.c +++ b/arch/arm64/kernel/kaslr.c @@ -106,7 +106,7 @@ u64 __init kaslr_early_init(void) * and supported. */ - if (arch_get_random_seed_long_early(&raw)) + if (arch_get_random_seed_words_early(&raw, 1)) seed ^= raw; if (!seed) { diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h index 25ba65df6b1a..bf2182f80480 100644 --- a/arch/powerpc/include/asm/archrandom.h +++ b/arch/powerpc/include/asm/archrandom.h @@ -4,34 +4,16 @@ #include <asm/machdep.h> -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { - return false; + return 0; } -static inline bool __must_check arch_get_random_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - if (ppc_md.get_random_seed) - return ppc_md.get_random_seed(v); - - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) -{ - unsigned long val; - bool rc; - - rc = arch_get_random_seed_long(&val); - if (rc) - *v = val; - - return rc; + if (words && ppc_md.get_random_seed && ppc_md.get_random_seed(v)) + return 1; + return 0; } #ifdef CONFIG_PPC_POWERNV diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index e08fb3124dca..18b2d80996b6 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -1207,7 +1207,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) break; #endif case H_RANDOM: - if (!arch_get_random_seed_long(&vcpu->arch.regs.gpr[4])) + if (!arch_get_random_seed_words(&vcpu->arch.regs.gpr[4], 1)) ret = H_HARDWARE; break; case H_RPT_INVALIDATE: diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h index 0a1c2e66c709..29f1a9bc3867 100644 --- a/arch/s390/include/asm/archrandom.h +++ b/arch/s390/include/asm/archrandom.h @@ -18,34 +18,19 @@ DECLARE_STATIC_KEY_FALSE(s390_arch_random_available); extern atomic64_t s390_arch_random_counter; -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { - return false; + return 0; } -static inline bool __must_check arch_get_random_int(unsigned int *v) -{ - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - if (static_branch_likely(&s390_arch_random_available)) { - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); - atomic64_add(sizeof(*v), &s390_arch_random_counter); - return true; - } - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { if (static_branch_likely(&s390_arch_random_available)) { - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); - atomic64_add(sizeof(*v), &s390_arch_random_counter); - return true; + cpacf_trng(NULL, 0, (u8 *)v, words * sizeof(*v)); + atomic64_add(words * sizeof(*v), &s390_arch_random_counter); + return words; } - return false; + return 0; } #endif /* _ASM_S390_ARCHRANDOM_H */ diff --git a/arch/um/include/asm/archrandom.h b/arch/um/include/asm/archrandom.h index 2f24cb96391d..6bcbd47fcb62 100644 --- a/arch/um/include/asm/archrandom.h +++ b/arch/um/include/asm/archrandom.h @@ -7,24 +7,19 @@ /* This is from <os.h>, but better not to #include that in a global header here. */ ssize_t os_getrandom(void *buf, size_t len, unsigned int flags); -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); -} + ssize_t ret; -static inline bool __must_check arch_get_random_int(unsigned int *v) -{ - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - return false; + ret = os_getrandom(v, words * sizeof(*v), 0); + if (ret < 0) + return 0; + return ret / sizeof(*v); } -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { - return false; + return 0; } #endif diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h index fb235b696175..a1717b81d876 100644 --- a/arch/x86/include/asm/archrandom.h +++ b/arch/x86/include/asm/archrandom.h @@ -31,20 +31,6 @@ static inline bool __must_check rdrand_long(unsigned long *v) return false; } -static inline bool __must_check rdrand_int(unsigned int *v) -{ - bool ok; - unsigned int retry = RDRAND_RETRY_LOOPS; - do { - asm volatile("rdrand %[out]" - CC_SET(c) - : CC_OUT(c) (ok), [out] "=r" (*v)); - if (ok) - return true; - } while (--retry); - return false; -} - static inline bool __must_check rdseed_long(unsigned long *v) { bool ok; @@ -54,38 +40,19 @@ static inline bool __must_check rdseed_long(unsigned long *v) return ok; } -static inline bool __must_check rdseed_int(unsigned int *v) -{ - bool ok; - asm volatile("rdseed %[out]" - CC_SET(c) - : CC_OUT(c) (ok), [out] "=r" (*v)); - return ok; -} - /* * These are the generic interfaces; they must not be declared if the * stubs in <linux/random.h> are to be invoked. */ -static inline bool __must_check arch_get_random_long(unsigned long *v) -{ - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_long(v) : false; -} - -static inline bool __must_check arch_get_random_int(unsigned int *v) -{ - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_int(v) : false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_long(v) : false; + return words && static_cpu_has(X86_FEATURE_RDRAND) && rdrand_long(v) ? 1 : 0; } -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_int(v) : false; + return words && static_cpu_has(X86_FEATURE_RDSEED) && rdseed_long(v) ? 1 : 0; } #ifndef CONFIG_UML diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c index 4fe7af58cfe1..f46c9ff3c0d4 100644 --- a/arch/x86/kernel/espfix_64.c +++ b/arch/x86/kernel/espfix_64.c @@ -100,7 +100,7 @@ static void init_espfix_random(void) * This is run before the entropy pools are initialized, * but this is hopefully better than nothing. */ - if (!arch_get_random_long(&rand)) { + if (!arch_get_random_words(&rand, 1)) { /* The constant is an arbitrary large prime */ rand = rdtsc(); rand *= 0xc345c6b72fd16123UL; diff --git a/drivers/char/random.c b/drivers/char/random.c index 0c6568ae5f68..70d8d1d7e2d7 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -596,12 +596,20 @@ static void extract_entropy(void *buf, size_t len) unsigned long rdseed[32 / sizeof(long)]; size_t counter; } block; - size_t i; + size_t i, words; - for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) { - if (!arch_get_random_seed_long(&block.rdseed[i]) && - !arch_get_random_long(&block.rdseed[i])) - block.rdseed[i] = random_get_entropy(); + for (i = 0; i < ARRAY_SIZE(block.rdseed);) { + words = arch_get_random_seed_words(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); + if (words) { + i += words; + continue; + } + words = arch_get_random_words(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); + if (words) { + i += words; + continue; + } + block.rdseed[i++] = random_get_entropy(); } spin_lock_irqsave(&input_pool.lock, flags); @@ -776,22 +784,31 @@ static struct notifier_block pm_notifier = { .notifier_call = random_pm_notifica int __init random_init(const char *command_line) { ktime_t now = ktime_get_real(); - unsigned int i, arch_bits; - unsigned long entropy; + size_t i, words, arch_bits; + unsigned long entropy[BLAKE2S_BLOCK_SIZE / sizeof(long)]; #if defined(LATENT_ENTROPY_PLUGIN) static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst __latent_entropy; _mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed)); #endif - for (i = 0, arch_bits = BLAKE2S_BLOCK_SIZE * 8; - i < BLAKE2S_BLOCK_SIZE; i += sizeof(entropy)) { - if (!arch_get_random_seed_long_early(&entropy) && - !arch_get_random_long_early(&entropy)) { - entropy = random_get_entropy(); - arch_bits -= sizeof(entropy) * 8; + for (i = 0, arch_bits = sizeof(entropy) * 8; i < ARRAY_SIZE(entropy);) { + words = arch_get_random_seed_words(entropy, ARRAY_SIZE(entropy) - i); + if (words) { + _mix_pool_bytes(entropy, sizeof(*entropy) * words); + i += words; + continue; } - _mix_pool_bytes(&entropy, sizeof(entropy)); + words = arch_get_random_words(entropy, ARRAY_SIZE(entropy) - i); + if (words) { + _mix_pool_bytes(entropy, sizeof(*entropy) * words); + i += words; + continue; + } + entropy[0] = random_get_entropy(); + _mix_pool_bytes(entropy, sizeof(*entropy)); + arch_bits -= sizeof(*entropy) * 8; + ++i; } _mix_pool_bytes(&now, sizeof(now)); _mix_pool_bytes(utsname(), sizeof(*(utsname()))); diff --git a/include/asm-generic/archrandom.h b/include/asm-generic/archrandom.h index 3a5ee202dd86..ae618916c74c 100644 --- a/include/asm-generic/archrandom.h +++ b/include/asm-generic/archrandom.h @@ -2,24 +2,14 @@ #ifndef __ASM_GENERIC_ARCHRANDOM_H__ #define __ASM_GENERIC_ARCHRANDOM_H__ -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) { - return false; + return 0; } -static inline bool __must_check arch_get_random_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) { - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) -{ - return false; + return 0; } #endif diff --git a/include/linux/random.h b/include/linux/random.h index 865770e29f3e..0a327a289f09 100644 --- a/include/linux/random.h +++ b/include/linux/random.h @@ -112,19 +112,19 @@ declare_get_random_var_wait(long, unsigned long) * Called from the boot CPU during startup; not valid to call once * secondary CPUs are up and preemption is possible. */ -#ifndef arch_get_random_seed_long_early -static inline bool __init arch_get_random_seed_long_early(unsigned long *v) +#ifndef arch_get_random_seed_words_early +static inline size_t __init arch_get_random_seed_words_early(unsigned long *v, size_t words) { WARN_ON(system_state != SYSTEM_BOOTING); - return arch_get_random_seed_long(v); + return arch_get_random_seed_words(v, words); } #endif -#ifndef arch_get_random_long_early -static inline bool __init arch_get_random_long_early(unsigned long *v) +#ifndef arch_get_random_words_early +static inline bool __init arch_get_random_words_early(unsigned long *v, size_t words) { WARN_ON(system_state != SYSTEM_BOOTING); - return arch_get_random_long(v); + return arch_get_random_words(v, words); } #endif -- 2.35.1 ^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH v2] random: handle archrandom in plural words 2022-07-17 20:03 ` [PATCH v2] " Jason A. Donenfeld @ 2022-07-18 6:31 ` Michael Ellerman 2022-07-18 6:46 ` Gabriel Paubert 2022-07-19 12:42 ` Mark Rutland 2022-07-22 8:08 ` [PATCH v2] random: handle archrandom in plural words Holger Dengler 2 siblings, 1 reply; 27+ messages in thread From: Michael Ellerman @ 2022-07-18 6:31 UTC (permalink / raw) To: Jason A. Donenfeld, linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86 Cc: Jason A. Donenfeld, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Harald Freudenberger "Jason A. Donenfeld" <Jason@zx2c4.com> writes: > The archrandom interface was originally designed for x86, which supplies > RDRAND/RDSEED for receiving random words into registers, resulting in > one function to generate an int and another to generate a long. However, > other architectures don't follow this. > > On arm64, the SMCCC TRNG interface can return between 1 and 3 words. On > s390, the CPACF TRNG interface can return between 1 and 32 words for the > same cost as for one word. On UML, the os_getrandom() interface can return > arbitrary amounts. > > So change the api signature to take a "words" parameter designating the > maximum number of words requested, and then return the number of words > generated. On powerpc a word is 32-bits and a doubleword is 64-bits (at least according to the ISA). I think that's also true on other 64-bit architectures. You could avoid any confusion by defining the API in terms of "longs" rather than "words". But that's just a comment, see what others think. > arch/powerpc/include/asm/archrandom.h | 30 ++------ > arch/powerpc/kvm/book3s_hv.c | 2 +- Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc) cheers ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2] random: handle archrandom in plural words 2022-07-18 6:31 ` Michael Ellerman @ 2022-07-18 6:46 ` Gabriel Paubert 2022-07-20 3:03 ` Michael Ellerman 0 siblings, 1 reply; 27+ messages in thread From: Gabriel Paubert @ 2022-07-18 6:46 UTC (permalink / raw) To: Michael Ellerman Cc: Jason A. Donenfeld, linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Catalin Marinas, Heiko Carstens, Johannes Berg, Harald Freudenberger, H . Peter Anvin, Alexander Gordeev, Borislav Petkov, Will Deacon, Thomas Gleixner On Mon, Jul 18, 2022 at 04:31:11PM +1000, Michael Ellerman wrote: > "Jason A. Donenfeld" <Jason@zx2c4.com> writes: > > The archrandom interface was originally designed for x86, which supplies > > RDRAND/RDSEED for receiving random words into registers, resulting in > > one function to generate an int and another to generate a long. However, > > other architectures don't follow this. > > > > On arm64, the SMCCC TRNG interface can return between 1 and 3 words. On > > s390, the CPACF TRNG interface can return between 1 and 32 words for the > > same cost as for one word. On UML, the os_getrandom() interface can return > > arbitrary amounts. > > > > So change the api signature to take a "words" parameter designating the > > maximum number of words requested, and then return the number of words > > generated. > > On powerpc a word is 32-bits and a doubleword is 64-bits (at least > according to the ISA). I think that's also true on other 64-bit > architectures. IIRC, this is (or was) not the case on Alpha, where word was defined as 16 bits. All assembly mnemonics had w for 16 bits, l for 32 bits, and q for 64 bits. Blame the PDP-11... Gabriel > > You could avoid any confusion by defining the API in terms of "longs" > rather than "words". > > But that's just a comment, see what others think. > > > arch/powerpc/include/asm/archrandom.h | 30 ++------ > > arch/powerpc/kvm/book3s_hv.c | 2 +- > > Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc) > > cheers ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2] random: handle archrandom in plural words 2022-07-18 6:46 ` Gabriel Paubert @ 2022-07-20 3:03 ` Michael Ellerman 0 siblings, 0 replies; 27+ messages in thread From: Michael Ellerman @ 2022-07-20 3:03 UTC (permalink / raw) To: Gabriel Paubert Cc: Jason A. Donenfeld, linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Catalin Marinas, Heiko Carstens, Johannes Berg, Harald Freudenberger, H . Peter Anvin, Alexander Gordeev, Borislav Petkov, Will Deacon, Thomas Gleixner Gabriel Paubert <paubert@iram.es> writes: > On Mon, Jul 18, 2022 at 04:31:11PM +1000, Michael Ellerman wrote: >> "Jason A. Donenfeld" <Jason@zx2c4.com> writes: >> > The archrandom interface was originally designed for x86, which supplies >> > RDRAND/RDSEED for receiving random words into registers, resulting in >> > one function to generate an int and another to generate a long. However, >> > other architectures don't follow this. >> > >> > On arm64, the SMCCC TRNG interface can return between 1 and 3 words. On >> > s390, the CPACF TRNG interface can return between 1 and 32 words for the >> > same cost as for one word. On UML, the os_getrandom() interface can return >> > arbitrary amounts. >> > >> > So change the api signature to take a "words" parameter designating the >> > maximum number of words requested, and then return the number of words >> > generated. >> >> On powerpc a word is 32-bits and a doubleword is 64-bits (at least >> according to the ISA). I think that's also true on other 64-bit >> architectures. > > IIRC, this is (or was) not the case on Alpha, where word was defined as > 16 bits. All assembly mnemonics had w for 16 bits, l for 32 bits, and q > for 64 bits. Yeah I should have said on *some* other 64-bit arches. Seems to be a common feature/hack on arches that have evolved over time, or been inspired by earlier arches. The latest Power ISA has octwords :) cheers ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2] random: handle archrandom in plural words 2022-07-17 20:03 ` [PATCH v2] " Jason A. Donenfeld 2022-07-18 6:31 ` Michael Ellerman @ 2022-07-19 12:42 ` Mark Rutland 2022-07-19 12:46 ` Jason A. Donenfeld 2022-07-22 8:08 ` [PATCH v2] random: handle archrandom in plural words Holger Dengler 2 siblings, 1 reply; 27+ messages in thread From: Mark Rutland @ 2022-07-19 12:42 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Michael Ellerman, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Harald Freudenberger On Sun, Jul 17, 2022 at 10:03:56PM +0200, Jason A. Donenfeld wrote: > The archrandom interface was originally designed for x86, which supplies > RDRAND/RDSEED for receiving random words into registers, resulting in > one function to generate an int and another to generate a long. However, > other architectures don't follow this. > > On arm64, the SMCCC TRNG interface can return between 1 and 3 words. On > s390, the CPACF TRNG interface can return between 1 and 32 words for the > same cost as for one word. On UML, the os_getrandom() interface can return > arbitrary amounts. > > So change the api signature to take a "words" parameter designating the > maximum number of words requested, and then return the number of words > generated. > > Since callers need to check this return value and loop anyway, each arch > implementation does not bother implementing its own loop to try again to > fill the requested number of words. Additionally, all existing callers > pass in a constant words parameter. Taken together, these two things > mean that the codegen doesn't really change much for one-word-at-a-time > platforms, while performance is greatly improved on platforms such as > s390. > > Cc: Will Deacon <will@kernel.org> > Cc: Michael Ellerman <mpe@ellerman.id.au> > Cc: Alexander Gordeev <agordeev@linux.ibm.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: H. Peter Anvin <hpa@zytor.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Johannes Berg <johannes@sipsolutions.net> > Cc: Harald Freudenberger <freude@linux.ibm.com> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> > --- > arch/arm64/include/asm/archrandom.h | 102 ++++++++++++-------------- > arch/arm64/kernel/kaslr.c | 2 +- > arch/powerpc/include/asm/archrandom.h | 30 ++------ > arch/powerpc/kvm/book3s_hv.c | 2 +- > arch/s390/include/asm/archrandom.h | 29 ++------ > arch/um/include/asm/archrandom.h | 21 ++---- > arch/x86/include/asm/archrandom.h | 41 +---------- > arch/x86/kernel/espfix_64.c | 2 +- > drivers/char/random.c | 45 ++++++++---- > include/asm-generic/archrandom.h | 18 +---- > include/linux/random.h | 12 +-- > 11 files changed, 116 insertions(+), 188 deletions(-) > > diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h > index c3b9fa56af67..7a24fdee3e2f 100644 > --- a/arch/arm64/include/asm/archrandom.h > +++ b/arch/arm64/include/asm/archrandom.h > @@ -58,7 +58,7 @@ static inline bool __arm64_rndrrs(unsigned long *v) > return ok; > } > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) As others have suggested, could we please s/words/longs/? That's a smaller change from the existing name, and avoids amibugity were "word" has an architecture-specific meaning. e.g. the ARM ARM (ARM DDI 0487G.b) defines: Word A 32-bit data item. Words are normally word-aligned in Arm systems. > { > /* > * Only support the generic interface after we have detected > @@ -66,27 +66,15 @@ static inline bool __must_check arch_get_random_long(unsigned long *v) > * cpufeature code and with potential scheduling between CPUs > * with and without the feature. > */ > - if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) > - return true; > - return false; > + if (words && cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) > + return 1; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) Likewise, s/words/longs/ woudl be clearer. > { > - if (cpus_have_const_cap(ARM64_HAS_RNG)) { > - unsigned long val; > - > - if (__arm64_rndr(&val)) { > - *v = val; > - return true; > - } > - } > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - struct arm_smccc_res res; > + if (!words) > + return 0; > > /* > * We prefer the SMCCC call, since its semantics (return actual > @@ -95,10 +83,23 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > * (the output of a pseudo RNG freshly seeded by a TRNG). > */ > if (smccc_trng_available) { > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); > + struct arm_smccc_res res; > + > + words = min_t(size_t, 3, words); > + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, words * 64, &res); > if ((int)res.a0 >= 0) { > - *v = res.a3; > - return true; > + switch (words) { > + case 3: > + *v++ = res.a1; > + fallthrough; > + case 2: > + *v++ = res.a2; > + fallthrough; > + case 1: > + *v++ = res.a3; > + break; > + } > + return words; > } > } > > @@ -108,32 +109,9 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > * enough to implement this API if no other entropy source exists. > */ > if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndrrs(v)) > - return true; > + return 1; > > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - struct arm_smccc_res res; > - unsigned long val; > - > - if (smccc_trng_available) { > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 32, &res); > - if ((int)res.a0 >= 0) { > - *v = res.a3 & GENMASK(31, 0); > - return true; > - } > - } > - > - if (cpus_have_const_cap(ARM64_HAS_RNG)) { > - if (__arm64_rndrrs(&val)) { > - *v = val; > - return true; > - } > - } > - > - return false; > + return 0; > } > > static inline bool __init __early_cpu_has_rndr(void) > @@ -143,26 +121,40 @@ static inline bool __init __early_cpu_has_rndr(void) > return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; > } > > -static inline bool __init __must_check > -arch_get_random_seed_long_early(unsigned long *v) > +static inline size_t __init __must_check > +arch_get_random_seed_words_early(unsigned long *v, size_t words) > { > WARN_ON(system_state != SYSTEM_BOOTING); > > + if (!words) > + return 0; > + > if (smccc_trng_available) { > struct arm_smccc_res res; > > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); > + words = min_t(size_t, 3, words); > + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, words * 64, &res); > if ((int)res.a0 >= 0) { > - *v = res.a3; > - return true; > + switch (words) { > + case 3: > + *v++ = res.a1; > + fallthrough; > + case 2: > + *v++ = res.a2; > + fallthrough; > + case 1: > + *v++ = res.a3; > + break; > + } > + return words; > } > } > > if (__early_cpu_has_rndr() && __arm64_rndr(v)) > - return true; > + return 1; > > - return false; > + return 0; > } > -#define arch_get_random_seed_long_early arch_get_random_seed_long_early > +#define arch_get_random_seed_words_early arch_get_random_seed_words_early > > #endif /* _ASM_ARCHRANDOM_H */ > diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c > index 418b2bba1521..ed77afe16121 100644 > --- a/arch/arm64/kernel/kaslr.c > +++ b/arch/arm64/kernel/kaslr.c > @@ -106,7 +106,7 @@ u64 __init kaslr_early_init(void) > * and supported. > */ > > - if (arch_get_random_seed_long_early(&raw)) > + if (arch_get_random_seed_words_early(&raw, 1)) > seed ^= raw; > > if (!seed) { FWIW, other than the naming nits, the arm64 bits look fine to me. Mark. > diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h > index 25ba65df6b1a..bf2182f80480 100644 > --- a/arch/powerpc/include/asm/archrandom.h > +++ b/arch/powerpc/include/asm/archrandom.h > @@ -4,34 +4,16 @@ > > #include <asm/machdep.h> > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - if (ppc_md.get_random_seed) > - return ppc_md.get_random_seed(v); > - > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - unsigned long val; > - bool rc; > - > - rc = arch_get_random_seed_long(&val); > - if (rc) > - *v = val; > - > - return rc; > + if (words && ppc_md.get_random_seed && ppc_md.get_random_seed(v)) > + return 1; > + return 0; > } > > #ifdef CONFIG_PPC_POWERNV > diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c > index e08fb3124dca..18b2d80996b6 100644 > --- a/arch/powerpc/kvm/book3s_hv.c > +++ b/arch/powerpc/kvm/book3s_hv.c > @@ -1207,7 +1207,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) > break; > #endif > case H_RANDOM: > - if (!arch_get_random_seed_long(&vcpu->arch.regs.gpr[4])) > + if (!arch_get_random_seed_words(&vcpu->arch.regs.gpr[4], 1)) > ret = H_HARDWARE; > break; > case H_RPT_INVALIDATE: > diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h > index 0a1c2e66c709..29f1a9bc3867 100644 > --- a/arch/s390/include/asm/archrandom.h > +++ b/arch/s390/include/asm/archrandom.h > @@ -18,34 +18,19 @@ > DECLARE_STATIC_KEY_FALSE(s390_arch_random_available); > extern atomic64_t s390_arch_random_counter; > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - if (static_branch_likely(&s390_arch_random_available)) { > - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); > - atomic64_add(sizeof(*v), &s390_arch_random_counter); > - return true; > - } > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > if (static_branch_likely(&s390_arch_random_available)) { > - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); > - atomic64_add(sizeof(*v), &s390_arch_random_counter); > - return true; > + cpacf_trng(NULL, 0, (u8 *)v, words * sizeof(*v)); > + atomic64_add(words * sizeof(*v), &s390_arch_random_counter); > + return words; > } > - return false; > + return 0; > } > > #endif /* _ASM_S390_ARCHRANDOM_H */ > diff --git a/arch/um/include/asm/archrandom.h b/arch/um/include/asm/archrandom.h > index 2f24cb96391d..6bcbd47fcb62 100644 > --- a/arch/um/include/asm/archrandom.h > +++ b/arch/um/include/asm/archrandom.h > @@ -7,24 +7,19 @@ > /* This is from <os.h>, but better not to #include that in a global header here. */ > ssize_t os_getrandom(void *buf, size_t len, unsigned int flags); > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); > -} > + ssize_t ret; > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - return false; > + ret = os_getrandom(v, words * sizeof(*v), 0); > + if (ret < 0) > + return 0; > + return ret / sizeof(*v); > } > > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return false; > + return 0; > } > > #endif > diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h > index fb235b696175..a1717b81d876 100644 > --- a/arch/x86/include/asm/archrandom.h > +++ b/arch/x86/include/asm/archrandom.h > @@ -31,20 +31,6 @@ static inline bool __must_check rdrand_long(unsigned long *v) > return false; > } > > -static inline bool __must_check rdrand_int(unsigned int *v) > -{ > - bool ok; > - unsigned int retry = RDRAND_RETRY_LOOPS; > - do { > - asm volatile("rdrand %[out]" > - CC_SET(c) > - : CC_OUT(c) (ok), [out] "=r" (*v)); > - if (ok) > - return true; > - } while (--retry); > - return false; > -} > - > static inline bool __must_check rdseed_long(unsigned long *v) > { > bool ok; > @@ -54,38 +40,19 @@ static inline bool __must_check rdseed_long(unsigned long *v) > return ok; > } > > -static inline bool __must_check rdseed_int(unsigned int *v) > -{ > - bool ok; > - asm volatile("rdseed %[out]" > - CC_SET(c) > - : CC_OUT(c) (ok), [out] "=r" (*v)); > - return ok; > -} > - > /* > * These are the generic interfaces; they must not be declared if the > * stubs in <linux/random.h> are to be invoked. > */ > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > -{ > - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_long(v) : false; > -} > - > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_int(v) : false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_long(v) : false; > + return words && static_cpu_has(X86_FEATURE_RDRAND) && rdrand_long(v) ? 1 : 0; > } > > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_int(v) : false; > + return words && static_cpu_has(X86_FEATURE_RDSEED) && rdseed_long(v) ? 1 : 0; > } > > #ifndef CONFIG_UML > diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c > index 4fe7af58cfe1..f46c9ff3c0d4 100644 > --- a/arch/x86/kernel/espfix_64.c > +++ b/arch/x86/kernel/espfix_64.c > @@ -100,7 +100,7 @@ static void init_espfix_random(void) > * This is run before the entropy pools are initialized, > * but this is hopefully better than nothing. > */ > - if (!arch_get_random_long(&rand)) { > + if (!arch_get_random_words(&rand, 1)) { > /* The constant is an arbitrary large prime */ > rand = rdtsc(); > rand *= 0xc345c6b72fd16123UL; > diff --git a/drivers/char/random.c b/drivers/char/random.c > index 0c6568ae5f68..70d8d1d7e2d7 100644 > --- a/drivers/char/random.c > +++ b/drivers/char/random.c > @@ -596,12 +596,20 @@ static void extract_entropy(void *buf, size_t len) > unsigned long rdseed[32 / sizeof(long)]; > size_t counter; > } block; > - size_t i; > + size_t i, words; > > - for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) { > - if (!arch_get_random_seed_long(&block.rdseed[i]) && > - !arch_get_random_long(&block.rdseed[i])) > - block.rdseed[i] = random_get_entropy(); > + for (i = 0; i < ARRAY_SIZE(block.rdseed);) { > + words = arch_get_random_seed_words(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); > + if (words) { > + i += words; > + continue; > + } > + words = arch_get_random_words(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); > + if (words) { > + i += words; > + continue; > + } > + block.rdseed[i++] = random_get_entropy(); > } > > spin_lock_irqsave(&input_pool.lock, flags); > @@ -776,22 +784,31 @@ static struct notifier_block pm_notifier = { .notifier_call = random_pm_notifica > int __init random_init(const char *command_line) > { > ktime_t now = ktime_get_real(); > - unsigned int i, arch_bits; > - unsigned long entropy; > + size_t i, words, arch_bits; > + unsigned long entropy[BLAKE2S_BLOCK_SIZE / sizeof(long)]; > > #if defined(LATENT_ENTROPY_PLUGIN) > static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst __latent_entropy; > _mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed)); > #endif > > - for (i = 0, arch_bits = BLAKE2S_BLOCK_SIZE * 8; > - i < BLAKE2S_BLOCK_SIZE; i += sizeof(entropy)) { > - if (!arch_get_random_seed_long_early(&entropy) && > - !arch_get_random_long_early(&entropy)) { > - entropy = random_get_entropy(); > - arch_bits -= sizeof(entropy) * 8; > + for (i = 0, arch_bits = sizeof(entropy) * 8; i < ARRAY_SIZE(entropy);) { > + words = arch_get_random_seed_words(entropy, ARRAY_SIZE(entropy) - i); > + if (words) { > + _mix_pool_bytes(entropy, sizeof(*entropy) * words); > + i += words; > + continue; > } > - _mix_pool_bytes(&entropy, sizeof(entropy)); > + words = arch_get_random_words(entropy, ARRAY_SIZE(entropy) - i); > + if (words) { > + _mix_pool_bytes(entropy, sizeof(*entropy) * words); > + i += words; > + continue; > + } > + entropy[0] = random_get_entropy(); > + _mix_pool_bytes(entropy, sizeof(*entropy)); > + arch_bits -= sizeof(*entropy) * 8; > + ++i; > } > _mix_pool_bytes(&now, sizeof(now)); > _mix_pool_bytes(utsname(), sizeof(*(utsname()))); > diff --git a/include/asm-generic/archrandom.h b/include/asm-generic/archrandom.h > index 3a5ee202dd86..ae618916c74c 100644 > --- a/include/asm-generic/archrandom.h > +++ b/include/asm-generic/archrandom.h > @@ -2,24 +2,14 @@ > #ifndef __ASM_GENERIC_ARCHRANDOM_H__ > #define __ASM_GENERIC_ARCHRANDOM_H__ > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - return false; > + return 0; > } > > #endif > diff --git a/include/linux/random.h b/include/linux/random.h > index 865770e29f3e..0a327a289f09 100644 > --- a/include/linux/random.h > +++ b/include/linux/random.h > @@ -112,19 +112,19 @@ declare_get_random_var_wait(long, unsigned long) > * Called from the boot CPU during startup; not valid to call once > * secondary CPUs are up and preemption is possible. > */ > -#ifndef arch_get_random_seed_long_early > -static inline bool __init arch_get_random_seed_long_early(unsigned long *v) > +#ifndef arch_get_random_seed_words_early > +static inline size_t __init arch_get_random_seed_words_early(unsigned long *v, size_t words) > { > WARN_ON(system_state != SYSTEM_BOOTING); > - return arch_get_random_seed_long(v); > + return arch_get_random_seed_words(v, words); > } > #endif > > -#ifndef arch_get_random_long_early > -static inline bool __init arch_get_random_long_early(unsigned long *v) > +#ifndef arch_get_random_words_early > +static inline bool __init arch_get_random_words_early(unsigned long *v, size_t words) > { > WARN_ON(system_state != SYSTEM_BOOTING); > - return arch_get_random_long(v); > + return arch_get_random_words(v, words); > } > #endif > > -- > 2.35.1 > ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2] random: handle archrandom in plural words 2022-07-19 12:42 ` Mark Rutland @ 2022-07-19 12:46 ` Jason A. Donenfeld 2022-07-19 13:02 ` [PATCH v3] random: handle archrandom with multiple longs Jason A. Donenfeld 0 siblings, 1 reply; 27+ messages in thread From: Jason A. Donenfeld @ 2022-07-19 12:46 UTC (permalink / raw) To: Mark Rutland Cc: LKML, linux-arm-kernel, PowerPC, linux-s390, X86 ML, Will Deacon, Michael Ellerman, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Harald Freudenberger Hi Mark, On Tue, Jul 19, 2022 at 2:42 PM Mark Rutland <mark.rutland@arm.com> wrote: > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > > As others have suggested, could we please s/words/longs/? That's a smaller > change from the existing name, and avoids amibugity were "word" has an > architecture-specific meaning. e.g. the ARM ARM (ARM DDI 0487G.b) defines: At first I had thought, "erm, I like 'words'" but decided to wait it out, and now it's two people with the same probably good objection to 'words', so I'll send a v+1 with 'longs' instead. Thanks for your feedback. Patch incoming. > FWIW, other than the naming nits, the arm64 bits look fine to me. Oh good. Can you send an Acked-by for v+1? Coming up shortly. Jason ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v3] random: handle archrandom with multiple longs 2022-07-19 12:46 ` Jason A. Donenfeld @ 2022-07-19 13:02 ` Jason A. Donenfeld 2022-07-19 13:17 ` Mark Rutland ` (6 more replies) 0 siblings, 7 replies; 27+ messages in thread From: Jason A. Donenfeld @ 2022-07-19 13:02 UTC (permalink / raw) To: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86 Cc: Jason A. Donenfeld, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Mark Rutland, Harald Freudenberger, Michael Ellerman The archrandom interface was originally designed for x86, which supplies RDRAND/RDSEED for receiving random words into registers, resulting in one function to generate an int and another to generate a long. However, other architectures don't follow this. On arm64, the SMCCC TRNG interface can return between 1 and 3 longs. On s390, the CPACF TRNG interface can return arbitrary amounts, with 32 longs having the same cost as one. On UML, the os_getrandom() interface can return arbitrary amounts. So change the api signature to take a "max_longs" parameter designating the maximum number of longs requested, and then return the number of longs generated. Since callers need to check this return value and loop anyway, each arch implementation does not bother implementing its own loop to try again to fill the maximum number of longs. Additionally, all existing callers pass in a constant max_longs parameter. Taken together, these two things mean that the codegen doesn't really change much for one-word-at-a-time platforms, while performance is greatly improved on platforms such as s390. Cc: Will Deacon <will@kernel.org> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Borislav Petkov <bp@suse.de> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Johannes Berg <johannes@sipsolutions.net> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Harald Freudenberger <freude@linux.ibm.com> Acked-by: Michael Ellerman <mpe@ellerman.id.au> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> --- arch/arm64/include/asm/archrandom.h | 102 ++++++++++++-------------- arch/arm64/kernel/kaslr.c | 2 +- arch/powerpc/include/asm/archrandom.h | 30 ++------ arch/powerpc/kvm/book3s_hv.c | 2 +- arch/s390/include/asm/archrandom.h | 29 ++------ arch/um/include/asm/archrandom.h | 21 ++---- arch/x86/include/asm/archrandom.h | 41 +---------- arch/x86/kernel/espfix_64.c | 2 +- drivers/char/random.c | 45 ++++++++---- include/asm-generic/archrandom.h | 18 +---- include/linux/random.h | 12 +-- 11 files changed, 116 insertions(+), 188 deletions(-) diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h index c3b9fa56af67..109e2a4454be 100644 --- a/arch/arm64/include/asm/archrandom.h +++ b/arch/arm64/include/asm/archrandom.h @@ -58,7 +58,7 @@ static inline bool __arm64_rndrrs(unsigned long *v) return ok; } -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { /* * Only support the generic interface after we have detected @@ -66,27 +66,15 @@ static inline bool __must_check arch_get_random_long(unsigned long *v) * cpufeature code and with potential scheduling between CPUs * with and without the feature. */ - if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) - return true; - return false; + if (max_longs && cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) + return 1; + return 0; } -static inline bool __must_check arch_get_random_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { - if (cpus_have_const_cap(ARM64_HAS_RNG)) { - unsigned long val; - - if (__arm64_rndr(&val)) { - *v = val; - return true; - } - } - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - struct arm_smccc_res res; + if (!max_longs) + return 0; /* * We prefer the SMCCC call, since its semantics (return actual @@ -95,10 +83,23 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) * (the output of a pseudo RNG freshly seeded by a TRNG). */ if (smccc_trng_available) { - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); + struct arm_smccc_res res; + + max_longs = min_t(size_t, 3, max_longs); + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, max_longs * 64, &res); if ((int)res.a0 >= 0) { - *v = res.a3; - return true; + switch (max_longs) { + case 3: + *v++ = res.a1; + fallthrough; + case 2: + *v++ = res.a2; + fallthrough; + case 1: + *v++ = res.a3; + break; + } + return max_longs; } } @@ -108,32 +109,9 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) * enough to implement this API if no other entropy source exists. */ if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndrrs(v)) - return true; + return 1; - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) -{ - struct arm_smccc_res res; - unsigned long val; - - if (smccc_trng_available) { - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 32, &res); - if ((int)res.a0 >= 0) { - *v = res.a3 & GENMASK(31, 0); - return true; - } - } - - if (cpus_have_const_cap(ARM64_HAS_RNG)) { - if (__arm64_rndrrs(&val)) { - *v = val; - return true; - } - } - - return false; + return 0; } static inline bool __init __early_cpu_has_rndr(void) @@ -143,26 +121,40 @@ static inline bool __init __early_cpu_has_rndr(void) return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; } -static inline bool __init __must_check -arch_get_random_seed_long_early(unsigned long *v) +static inline size_t __init __must_check +arch_get_random_seed_longs_early(unsigned long *v, size_t max_longs) { WARN_ON(system_state != SYSTEM_BOOTING); + if (!max_longs) + return 0; + if (smccc_trng_available) { struct arm_smccc_res res; - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); + max_longs = min_t(size_t, 3, max_longs); + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, max_longs * 64, &res); if ((int)res.a0 >= 0) { - *v = res.a3; - return true; + switch (max_longs) { + case 3: + *v++ = res.a1; + fallthrough; + case 2: + *v++ = res.a2; + fallthrough; + case 1: + *v++ = res.a3; + break; + } + return max_longs; } } if (__early_cpu_has_rndr() && __arm64_rndr(v)) - return true; + return 1; - return false; + return 0; } -#define arch_get_random_seed_long_early arch_get_random_seed_long_early +#define arch_get_random_seed_longs_early arch_get_random_seed_longs_early #endif /* _ASM_ARCHRANDOM_H */ diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c index 418b2bba1521..c5d541f358d3 100644 --- a/arch/arm64/kernel/kaslr.c +++ b/arch/arm64/kernel/kaslr.c @@ -106,7 +106,7 @@ u64 __init kaslr_early_init(void) * and supported. */ - if (arch_get_random_seed_long_early(&raw)) + if (arch_get_random_seed_longs_early(&raw, 1)) seed ^= raw; if (!seed) { diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h index 25ba65df6b1a..0e365c5b2396 100644 --- a/arch/powerpc/include/asm/archrandom.h +++ b/arch/powerpc/include/asm/archrandom.h @@ -4,34 +4,16 @@ #include <asm/machdep.h> -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { - return false; + return 0; } -static inline bool __must_check arch_get_random_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - if (ppc_md.get_random_seed) - return ppc_md.get_random_seed(v); - - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) -{ - unsigned long val; - bool rc; - - rc = arch_get_random_seed_long(&val); - if (rc) - *v = val; - - return rc; + if (max_longs && ppc_md.get_random_seed && ppc_md.get_random_seed(v)) + return 1; + return 0; } #ifdef CONFIG_PPC_POWERNV diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index e08fb3124dca..631062cde6b4 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -1207,7 +1207,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) break; #endif case H_RANDOM: - if (!arch_get_random_seed_long(&vcpu->arch.regs.gpr[4])) + if (!arch_get_random_seed_longs(&vcpu->arch.regs.gpr[4], 1)) ret = H_HARDWARE; break; case H_RPT_INVALIDATE: diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h index 0a1c2e66c709..cf5e000df0a1 100644 --- a/arch/s390/include/asm/archrandom.h +++ b/arch/s390/include/asm/archrandom.h @@ -18,34 +18,19 @@ DECLARE_STATIC_KEY_FALSE(s390_arch_random_available); extern atomic64_t s390_arch_random_counter; -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { - return false; + return 0; } -static inline bool __must_check arch_get_random_int(unsigned int *v) -{ - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - if (static_branch_likely(&s390_arch_random_available)) { - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); - atomic64_add(sizeof(*v), &s390_arch_random_counter); - return true; - } - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { if (static_branch_likely(&s390_arch_random_available)) { - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); - atomic64_add(sizeof(*v), &s390_arch_random_counter); - return true; + cpacf_trng(NULL, 0, (u8 *)v, max_longs * sizeof(*v)); + atomic64_add(max_longs * sizeof(*v), &s390_arch_random_counter); + return max_longs; } - return false; + return 0; } #endif /* _ASM_S390_ARCHRANDOM_H */ diff --git a/arch/um/include/asm/archrandom.h b/arch/um/include/asm/archrandom.h index 2f24cb96391d..24e16c979c51 100644 --- a/arch/um/include/asm/archrandom.h +++ b/arch/um/include/asm/archrandom.h @@ -7,24 +7,19 @@ /* This is from <os.h>, but better not to #include that in a global header here. */ ssize_t os_getrandom(void *buf, size_t len, unsigned int flags); -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); -} + ssize_t ret; -static inline bool __must_check arch_get_random_int(unsigned int *v) -{ - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - return false; + ret = os_getrandom(v, max_longs * sizeof(*v), 0); + if (ret < 0) + return 0; + return ret / sizeof(*v); } -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { - return false; + return 0; } #endif diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h index fb235b696175..02bae8e0758b 100644 --- a/arch/x86/include/asm/archrandom.h +++ b/arch/x86/include/asm/archrandom.h @@ -31,20 +31,6 @@ static inline bool __must_check rdrand_long(unsigned long *v) return false; } -static inline bool __must_check rdrand_int(unsigned int *v) -{ - bool ok; - unsigned int retry = RDRAND_RETRY_LOOPS; - do { - asm volatile("rdrand %[out]" - CC_SET(c) - : CC_OUT(c) (ok), [out] "=r" (*v)); - if (ok) - return true; - } while (--retry); - return false; -} - static inline bool __must_check rdseed_long(unsigned long *v) { bool ok; @@ -54,38 +40,19 @@ static inline bool __must_check rdseed_long(unsigned long *v) return ok; } -static inline bool __must_check rdseed_int(unsigned int *v) -{ - bool ok; - asm volatile("rdseed %[out]" - CC_SET(c) - : CC_OUT(c) (ok), [out] "=r" (*v)); - return ok; -} - /* * These are the generic interfaces; they must not be declared if the * stubs in <linux/random.h> are to be invoked. */ -static inline bool __must_check arch_get_random_long(unsigned long *v) -{ - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_long(v) : false; -} - -static inline bool __must_check arch_get_random_int(unsigned int *v) -{ - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_int(v) : false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_long(v) : false; + return max_longs && static_cpu_has(X86_FEATURE_RDRAND) && rdrand_long(v) ? 1 : 0; } -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_int(v) : false; + return max_longs && static_cpu_has(X86_FEATURE_RDSEED) && rdseed_long(v) ? 1 : 0; } #ifndef CONFIG_UML diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c index 4fe7af58cfe1..9417d5aa7305 100644 --- a/arch/x86/kernel/espfix_64.c +++ b/arch/x86/kernel/espfix_64.c @@ -100,7 +100,7 @@ static void init_espfix_random(void) * This is run before the entropy pools are initialized, * but this is hopefully better than nothing. */ - if (!arch_get_random_long(&rand)) { + if (!arch_get_random_longs(&rand, 1)) { /* The constant is an arbitrary large prime */ rand = rdtsc(); rand *= 0xc345c6b72fd16123UL; diff --git a/drivers/char/random.c b/drivers/char/random.c index 0c6568ae5f68..7bf11fa66265 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -596,12 +596,20 @@ static void extract_entropy(void *buf, size_t len) unsigned long rdseed[32 / sizeof(long)]; size_t counter; } block; - size_t i; + size_t i, longs; - for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) { - if (!arch_get_random_seed_long(&block.rdseed[i]) && - !arch_get_random_long(&block.rdseed[i])) - block.rdseed[i] = random_get_entropy(); + for (i = 0; i < ARRAY_SIZE(block.rdseed);) { + longs = arch_get_random_seed_longs(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); + if (longs) { + i += longs; + continue; + } + longs = arch_get_random_longs(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); + if (longs) { + i += longs; + continue; + } + block.rdseed[i++] = random_get_entropy(); } spin_lock_irqsave(&input_pool.lock, flags); @@ -776,22 +784,31 @@ static struct notifier_block pm_notifier = { .notifier_call = random_pm_notifica int __init random_init(const char *command_line) { ktime_t now = ktime_get_real(); - unsigned int i, arch_bits; - unsigned long entropy; + size_t i, longs, arch_bits; + unsigned long entropy[BLAKE2S_BLOCK_SIZE / sizeof(long)]; #if defined(LATENT_ENTROPY_PLUGIN) static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst __latent_entropy; _mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed)); #endif - for (i = 0, arch_bits = BLAKE2S_BLOCK_SIZE * 8; - i < BLAKE2S_BLOCK_SIZE; i += sizeof(entropy)) { - if (!arch_get_random_seed_long_early(&entropy) && - !arch_get_random_long_early(&entropy)) { - entropy = random_get_entropy(); - arch_bits -= sizeof(entropy) * 8; + for (i = 0, arch_bits = sizeof(entropy) * 8; i < ARRAY_SIZE(entropy);) { + longs = arch_get_random_seed_longs(entropy, ARRAY_SIZE(entropy) - i); + if (longs) { + _mix_pool_bytes(entropy, sizeof(*entropy) * longs); + i += longs; + continue; } - _mix_pool_bytes(&entropy, sizeof(entropy)); + longs = arch_get_random_longs(entropy, ARRAY_SIZE(entropy) - i); + if (longs) { + _mix_pool_bytes(entropy, sizeof(*entropy) * longs); + i += longs; + continue; + } + entropy[0] = random_get_entropy(); + _mix_pool_bytes(entropy, sizeof(*entropy)); + arch_bits -= sizeof(*entropy) * 8; + ++i; } _mix_pool_bytes(&now, sizeof(now)); _mix_pool_bytes(utsname(), sizeof(*(utsname()))); diff --git a/include/asm-generic/archrandom.h b/include/asm-generic/archrandom.h index 3a5ee202dd86..3cd7f980cfdc 100644 --- a/include/asm-generic/archrandom.h +++ b/include/asm-generic/archrandom.h @@ -2,24 +2,14 @@ #ifndef __ASM_GENERIC_ARCHRANDOM_H__ #define __ASM_GENERIC_ARCHRANDOM_H__ -static inline bool __must_check arch_get_random_long(unsigned long *v) +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { - return false; + return 0; } -static inline bool __must_check arch_get_random_int(unsigned int *v) +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { - return false; -} - -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) -{ - return false; -} - -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) -{ - return false; + return 0; } #endif diff --git a/include/linux/random.h b/include/linux/random.h index 865770e29f3e..3fec206487f6 100644 --- a/include/linux/random.h +++ b/include/linux/random.h @@ -112,19 +112,19 @@ declare_get_random_var_wait(long, unsigned long) * Called from the boot CPU during startup; not valid to call once * secondary CPUs are up and preemption is possible. */ -#ifndef arch_get_random_seed_long_early -static inline bool __init arch_get_random_seed_long_early(unsigned long *v) +#ifndef arch_get_random_seed_longs_early +static inline size_t __init arch_get_random_seed_longs_early(unsigned long *v, size_t max_longs) { WARN_ON(system_state != SYSTEM_BOOTING); - return arch_get_random_seed_long(v); + return arch_get_random_seed_longs(v, max_longs); } #endif -#ifndef arch_get_random_long_early -static inline bool __init arch_get_random_long_early(unsigned long *v) +#ifndef arch_get_random_longs_early +static inline bool __init arch_get_random_longs_early(unsigned long *v, size_t max_longs) { WARN_ON(system_state != SYSTEM_BOOTING); - return arch_get_random_long(v); + return arch_get_random_longs(v, max_longs); } #endif -- 2.35.1 ^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH v3] random: handle archrandom with multiple longs 2022-07-19 13:02 ` [PATCH v3] random: handle archrandom with multiple longs Jason A. Donenfeld @ 2022-07-19 13:17 ` Mark Rutland 2022-07-19 13:48 ` Catalin Marinas ` (5 subsequent siblings) 6 siblings, 0 replies; 27+ messages in thread From: Mark Rutland @ 2022-07-19 13:17 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Harald Freudenberger, Michael Ellerman On Tue, Jul 19, 2022 at 03:02:07PM +0200, Jason A. Donenfeld wrote: > The archrandom interface was originally designed for x86, which supplies > RDRAND/RDSEED for receiving random words into registers, resulting in > one function to generate an int and another to generate a long. However, > other architectures don't follow this. > > On arm64, the SMCCC TRNG interface can return between 1 and 3 longs. On > s390, the CPACF TRNG interface can return arbitrary amounts, with 32 > longs having the same cost as one. On UML, the os_getrandom() interface > can return arbitrary amounts. > > So change the api signature to take a "max_longs" parameter designating > the maximum number of longs requested, and then return the number of > longs generated. > > Since callers need to check this return value and loop anyway, each arch > implementation does not bother implementing its own loop to try again to > fill the maximum number of longs. Additionally, all existing callers > pass in a constant max_longs parameter. Taken together, these two things > mean that the codegen doesn't really change much for one-word-at-a-time > platforms, while performance is greatly improved on platforms such as > s390. > > Cc: Will Deacon <will@kernel.org> > Cc: Alexander Gordeev <agordeev@linux.ibm.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: H. Peter Anvin <hpa@zytor.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Johannes Berg <johannes@sipsolutions.net> > Cc: Mark Rutland <mark.rutland@arm.com> > Cc: Harald Freudenberger <freude@linux.ibm.com> > Acked-by: Michael Ellerman <mpe@ellerman.id.au> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> The arm64 bits looks good to me, so FWIW: Acked-by: Mark Rutland <mark.rutland@arm.com> Mark. > --- > arch/arm64/include/asm/archrandom.h | 102 ++++++++++++-------------- > arch/arm64/kernel/kaslr.c | 2 +- > arch/powerpc/include/asm/archrandom.h | 30 ++------ > arch/powerpc/kvm/book3s_hv.c | 2 +- > arch/s390/include/asm/archrandom.h | 29 ++------ > arch/um/include/asm/archrandom.h | 21 ++---- > arch/x86/include/asm/archrandom.h | 41 +---------- > arch/x86/kernel/espfix_64.c | 2 +- > drivers/char/random.c | 45 ++++++++---- > include/asm-generic/archrandom.h | 18 +---- > include/linux/random.h | 12 +-- > 11 files changed, 116 insertions(+), 188 deletions(-) > > diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h > index c3b9fa56af67..109e2a4454be 100644 > --- a/arch/arm64/include/asm/archrandom.h > +++ b/arch/arm64/include/asm/archrandom.h > @@ -58,7 +58,7 @@ static inline bool __arm64_rndrrs(unsigned long *v) > return ok; > } > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > /* > * Only support the generic interface after we have detected > @@ -66,27 +66,15 @@ static inline bool __must_check arch_get_random_long(unsigned long *v) > * cpufeature code and with potential scheduling between CPUs > * with and without the feature. > */ > - if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) > - return true; > - return false; > + if (max_longs && cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) > + return 1; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > - if (cpus_have_const_cap(ARM64_HAS_RNG)) { > - unsigned long val; > - > - if (__arm64_rndr(&val)) { > - *v = val; > - return true; > - } > - } > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - struct arm_smccc_res res; > + if (!max_longs) > + return 0; > > /* > * We prefer the SMCCC call, since its semantics (return actual > @@ -95,10 +83,23 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > * (the output of a pseudo RNG freshly seeded by a TRNG). > */ > if (smccc_trng_available) { > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); > + struct arm_smccc_res res; > + > + max_longs = min_t(size_t, 3, max_longs); > + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, max_longs * 64, &res); > if ((int)res.a0 >= 0) { > - *v = res.a3; > - return true; > + switch (max_longs) { > + case 3: > + *v++ = res.a1; > + fallthrough; > + case 2: > + *v++ = res.a2; > + fallthrough; > + case 1: > + *v++ = res.a3; > + break; > + } > + return max_longs; > } > } > > @@ -108,32 +109,9 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > * enough to implement this API if no other entropy source exists. > */ > if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndrrs(v)) > - return true; > + return 1; > > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - struct arm_smccc_res res; > - unsigned long val; > - > - if (smccc_trng_available) { > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 32, &res); > - if ((int)res.a0 >= 0) { > - *v = res.a3 & GENMASK(31, 0); > - return true; > - } > - } > - > - if (cpus_have_const_cap(ARM64_HAS_RNG)) { > - if (__arm64_rndrrs(&val)) { > - *v = val; > - return true; > - } > - } > - > - return false; > + return 0; > } > > static inline bool __init __early_cpu_has_rndr(void) > @@ -143,26 +121,40 @@ static inline bool __init __early_cpu_has_rndr(void) > return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; > } > > -static inline bool __init __must_check > -arch_get_random_seed_long_early(unsigned long *v) > +static inline size_t __init __must_check > +arch_get_random_seed_longs_early(unsigned long *v, size_t max_longs) > { > WARN_ON(system_state != SYSTEM_BOOTING); > > + if (!max_longs) > + return 0; > + > if (smccc_trng_available) { > struct arm_smccc_res res; > > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); > + max_longs = min_t(size_t, 3, max_longs); > + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, max_longs * 64, &res); > if ((int)res.a0 >= 0) { > - *v = res.a3; > - return true; > + switch (max_longs) { > + case 3: > + *v++ = res.a1; > + fallthrough; > + case 2: > + *v++ = res.a2; > + fallthrough; > + case 1: > + *v++ = res.a3; > + break; > + } > + return max_longs; > } > } > > if (__early_cpu_has_rndr() && __arm64_rndr(v)) > - return true; > + return 1; > > - return false; > + return 0; > } > -#define arch_get_random_seed_long_early arch_get_random_seed_long_early > +#define arch_get_random_seed_longs_early arch_get_random_seed_longs_early > > #endif /* _ASM_ARCHRANDOM_H */ > diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c > index 418b2bba1521..c5d541f358d3 100644 > --- a/arch/arm64/kernel/kaslr.c > +++ b/arch/arm64/kernel/kaslr.c > @@ -106,7 +106,7 @@ u64 __init kaslr_early_init(void) > * and supported. > */ > > - if (arch_get_random_seed_long_early(&raw)) > + if (arch_get_random_seed_longs_early(&raw, 1)) > seed ^= raw; > > if (!seed) { > diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h > index 25ba65df6b1a..0e365c5b2396 100644 > --- a/arch/powerpc/include/asm/archrandom.h > +++ b/arch/powerpc/include/asm/archrandom.h > @@ -4,34 +4,16 @@ > > #include <asm/machdep.h> > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - if (ppc_md.get_random_seed) > - return ppc_md.get_random_seed(v); > - > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - unsigned long val; > - bool rc; > - > - rc = arch_get_random_seed_long(&val); > - if (rc) > - *v = val; > - > - return rc; > + if (max_longs && ppc_md.get_random_seed && ppc_md.get_random_seed(v)) > + return 1; > + return 0; > } > > #ifdef CONFIG_PPC_POWERNV > diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c > index e08fb3124dca..631062cde6b4 100644 > --- a/arch/powerpc/kvm/book3s_hv.c > +++ b/arch/powerpc/kvm/book3s_hv.c > @@ -1207,7 +1207,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) > break; > #endif > case H_RANDOM: > - if (!arch_get_random_seed_long(&vcpu->arch.regs.gpr[4])) > + if (!arch_get_random_seed_longs(&vcpu->arch.regs.gpr[4], 1)) > ret = H_HARDWARE; > break; > case H_RPT_INVALIDATE: > diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h > index 0a1c2e66c709..cf5e000df0a1 100644 > --- a/arch/s390/include/asm/archrandom.h > +++ b/arch/s390/include/asm/archrandom.h > @@ -18,34 +18,19 @@ > DECLARE_STATIC_KEY_FALSE(s390_arch_random_available); > extern atomic64_t s390_arch_random_counter; > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - if (static_branch_likely(&s390_arch_random_available)) { > - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); > - atomic64_add(sizeof(*v), &s390_arch_random_counter); > - return true; > - } > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > if (static_branch_likely(&s390_arch_random_available)) { > - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); > - atomic64_add(sizeof(*v), &s390_arch_random_counter); > - return true; > + cpacf_trng(NULL, 0, (u8 *)v, max_longs * sizeof(*v)); > + atomic64_add(max_longs * sizeof(*v), &s390_arch_random_counter); > + return max_longs; > } > - return false; > + return 0; > } > > #endif /* _ASM_S390_ARCHRANDOM_H */ > diff --git a/arch/um/include/asm/archrandom.h b/arch/um/include/asm/archrandom.h > index 2f24cb96391d..24e16c979c51 100644 > --- a/arch/um/include/asm/archrandom.h > +++ b/arch/um/include/asm/archrandom.h > @@ -7,24 +7,19 @@ > /* This is from <os.h>, but better not to #include that in a global header here. */ > ssize_t os_getrandom(void *buf, size_t len, unsigned int flags); > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); > -} > + ssize_t ret; > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - return false; > + ret = os_getrandom(v, max_longs * sizeof(*v), 0); > + if (ret < 0) > + return 0; > + return ret / sizeof(*v); > } > > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > - return false; > + return 0; > } > > #endif > diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h > index fb235b696175..02bae8e0758b 100644 > --- a/arch/x86/include/asm/archrandom.h > +++ b/arch/x86/include/asm/archrandom.h > @@ -31,20 +31,6 @@ static inline bool __must_check rdrand_long(unsigned long *v) > return false; > } > > -static inline bool __must_check rdrand_int(unsigned int *v) > -{ > - bool ok; > - unsigned int retry = RDRAND_RETRY_LOOPS; > - do { > - asm volatile("rdrand %[out]" > - CC_SET(c) > - : CC_OUT(c) (ok), [out] "=r" (*v)); > - if (ok) > - return true; > - } while (--retry); > - return false; > -} > - > static inline bool __must_check rdseed_long(unsigned long *v) > { > bool ok; > @@ -54,38 +40,19 @@ static inline bool __must_check rdseed_long(unsigned long *v) > return ok; > } > > -static inline bool __must_check rdseed_int(unsigned int *v) > -{ > - bool ok; > - asm volatile("rdseed %[out]" > - CC_SET(c) > - : CC_OUT(c) (ok), [out] "=r" (*v)); > - return ok; > -} > - > /* > * These are the generic interfaces; they must not be declared if the > * stubs in <linux/random.h> are to be invoked. > */ > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > -{ > - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_long(v) : false; > -} > - > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_int(v) : false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_long(v) : false; > + return max_longs && static_cpu_has(X86_FEATURE_RDRAND) && rdrand_long(v) ? 1 : 0; > } > > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_int(v) : false; > + return max_longs && static_cpu_has(X86_FEATURE_RDSEED) && rdseed_long(v) ? 1 : 0; > } > > #ifndef CONFIG_UML > diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c > index 4fe7af58cfe1..9417d5aa7305 100644 > --- a/arch/x86/kernel/espfix_64.c > +++ b/arch/x86/kernel/espfix_64.c > @@ -100,7 +100,7 @@ static void init_espfix_random(void) > * This is run before the entropy pools are initialized, > * but this is hopefully better than nothing. > */ > - if (!arch_get_random_long(&rand)) { > + if (!arch_get_random_longs(&rand, 1)) { > /* The constant is an arbitrary large prime */ > rand = rdtsc(); > rand *= 0xc345c6b72fd16123UL; > diff --git a/drivers/char/random.c b/drivers/char/random.c > index 0c6568ae5f68..7bf11fa66265 100644 > --- a/drivers/char/random.c > +++ b/drivers/char/random.c > @@ -596,12 +596,20 @@ static void extract_entropy(void *buf, size_t len) > unsigned long rdseed[32 / sizeof(long)]; > size_t counter; > } block; > - size_t i; > + size_t i, longs; > > - for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) { > - if (!arch_get_random_seed_long(&block.rdseed[i]) && > - !arch_get_random_long(&block.rdseed[i])) > - block.rdseed[i] = random_get_entropy(); > + for (i = 0; i < ARRAY_SIZE(block.rdseed);) { > + longs = arch_get_random_seed_longs(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); > + if (longs) { > + i += longs; > + continue; > + } > + longs = arch_get_random_longs(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); > + if (longs) { > + i += longs; > + continue; > + } > + block.rdseed[i++] = random_get_entropy(); > } > > spin_lock_irqsave(&input_pool.lock, flags); > @@ -776,22 +784,31 @@ static struct notifier_block pm_notifier = { .notifier_call = random_pm_notifica > int __init random_init(const char *command_line) > { > ktime_t now = ktime_get_real(); > - unsigned int i, arch_bits; > - unsigned long entropy; > + size_t i, longs, arch_bits; > + unsigned long entropy[BLAKE2S_BLOCK_SIZE / sizeof(long)]; > > #if defined(LATENT_ENTROPY_PLUGIN) > static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst __latent_entropy; > _mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed)); > #endif > > - for (i = 0, arch_bits = BLAKE2S_BLOCK_SIZE * 8; > - i < BLAKE2S_BLOCK_SIZE; i += sizeof(entropy)) { > - if (!arch_get_random_seed_long_early(&entropy) && > - !arch_get_random_long_early(&entropy)) { > - entropy = random_get_entropy(); > - arch_bits -= sizeof(entropy) * 8; > + for (i = 0, arch_bits = sizeof(entropy) * 8; i < ARRAY_SIZE(entropy);) { > + longs = arch_get_random_seed_longs(entropy, ARRAY_SIZE(entropy) - i); > + if (longs) { > + _mix_pool_bytes(entropy, sizeof(*entropy) * longs); > + i += longs; > + continue; > } > - _mix_pool_bytes(&entropy, sizeof(entropy)); > + longs = arch_get_random_longs(entropy, ARRAY_SIZE(entropy) - i); > + if (longs) { > + _mix_pool_bytes(entropy, sizeof(*entropy) * longs); > + i += longs; > + continue; > + } > + entropy[0] = random_get_entropy(); > + _mix_pool_bytes(entropy, sizeof(*entropy)); > + arch_bits -= sizeof(*entropy) * 8; > + ++i; > } > _mix_pool_bytes(&now, sizeof(now)); > _mix_pool_bytes(utsname(), sizeof(*(utsname()))); > diff --git a/include/asm-generic/archrandom.h b/include/asm-generic/archrandom.h > index 3a5ee202dd86..3cd7f980cfdc 100644 > --- a/include/asm-generic/archrandom.h > +++ b/include/asm-generic/archrandom.h > @@ -2,24 +2,14 @@ > #ifndef __ASM_GENERIC_ARCHRANDOM_H__ > #define __ASM_GENERIC_ARCHRANDOM_H__ > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - return false; > + return 0; > } > > #endif > diff --git a/include/linux/random.h b/include/linux/random.h > index 865770e29f3e..3fec206487f6 100644 > --- a/include/linux/random.h > +++ b/include/linux/random.h > @@ -112,19 +112,19 @@ declare_get_random_var_wait(long, unsigned long) > * Called from the boot CPU during startup; not valid to call once > * secondary CPUs are up and preemption is possible. > */ > -#ifndef arch_get_random_seed_long_early > -static inline bool __init arch_get_random_seed_long_early(unsigned long *v) > +#ifndef arch_get_random_seed_longs_early > +static inline size_t __init arch_get_random_seed_longs_early(unsigned long *v, size_t max_longs) > { > WARN_ON(system_state != SYSTEM_BOOTING); > - return arch_get_random_seed_long(v); > + return arch_get_random_seed_longs(v, max_longs); > } > #endif > > -#ifndef arch_get_random_long_early > -static inline bool __init arch_get_random_long_early(unsigned long *v) > +#ifndef arch_get_random_longs_early > +static inline bool __init arch_get_random_longs_early(unsigned long *v, size_t max_longs) > { > WARN_ON(system_state != SYSTEM_BOOTING); > - return arch_get_random_long(v); > + return arch_get_random_longs(v, max_longs); > } > #endif > > -- > 2.35.1 > ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v3] random: handle archrandom with multiple longs 2022-07-19 13:02 ` [PATCH v3] random: handle archrandom with multiple longs Jason A. Donenfeld 2022-07-19 13:17 ` Mark Rutland @ 2022-07-19 13:48 ` Catalin Marinas 2022-07-22 12:06 ` Heiko Carstens ` (4 subsequent siblings) 6 siblings, 0 replies; 27+ messages in thread From: Catalin Marinas @ 2022-07-19 13:48 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Borislav Petkov, Heiko Carstens, Johannes Berg, Mark Rutland, Harald Freudenberger, Michael Ellerman On Tue, Jul 19, 2022 at 03:02:07PM +0200, Jason A. Donenfeld wrote: > The archrandom interface was originally designed for x86, which supplies > RDRAND/RDSEED for receiving random words into registers, resulting in > one function to generate an int and another to generate a long. However, > other architectures don't follow this. > > On arm64, the SMCCC TRNG interface can return between 1 and 3 longs. On > s390, the CPACF TRNG interface can return arbitrary amounts, with 32 > longs having the same cost as one. On UML, the os_getrandom() interface > can return arbitrary amounts. > > So change the api signature to take a "max_longs" parameter designating > the maximum number of longs requested, and then return the number of > longs generated. > > Since callers need to check this return value and loop anyway, each arch > implementation does not bother implementing its own loop to try again to > fill the maximum number of longs. Additionally, all existing callers > pass in a constant max_longs parameter. Taken together, these two things > mean that the codegen doesn't really change much for one-word-at-a-time > platforms, while performance is greatly improved on platforms such as > s390. > > Cc: Will Deacon <will@kernel.org> > Cc: Alexander Gordeev <agordeev@linux.ibm.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: H. Peter Anvin <hpa@zytor.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Johannes Berg <johannes@sipsolutions.net> > Cc: Mark Rutland <mark.rutland@arm.com> > Cc: Harald Freudenberger <freude@linux.ibm.com> > Acked-by: Michael Ellerman <mpe@ellerman.id.au> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> For arm64: Acked-by: Catalin Marinas <catalin.marinas@arm.com> ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v3] random: handle archrandom with multiple longs 2022-07-19 13:02 ` [PATCH v3] random: handle archrandom with multiple longs Jason A. Donenfeld 2022-07-19 13:17 ` Mark Rutland 2022-07-19 13:48 ` Catalin Marinas @ 2022-07-22 12:06 ` Heiko Carstens 2022-07-24 22:47 ` Jason A. Donenfeld ` (3 subsequent siblings) 6 siblings, 0 replies; 27+ messages in thread From: Heiko Carstens @ 2022-07-22 12:06 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Johannes Berg, Mark Rutland, Harald Freudenberger, Michael Ellerman, Holger Dengler On Tue, Jul 19, 2022 at 03:02:07PM +0200, Jason A. Donenfeld wrote: > The archrandom interface was originally designed for x86, which supplies > RDRAND/RDSEED for receiving random words into registers, resulting in > one function to generate an int and another to generate a long. However, > other architectures don't follow this. > > On arm64, the SMCCC TRNG interface can return between 1 and 3 longs. On > s390, the CPACF TRNG interface can return arbitrary amounts, with 32 > longs having the same cost as one. On UML, the os_getrandom() interface > can return arbitrary amounts. > > So change the api signature to take a "max_longs" parameter designating > the maximum number of longs requested, and then return the number of > longs generated. > > Since callers need to check this return value and loop anyway, each arch > implementation does not bother implementing its own loop to try again to > fill the maximum number of longs. Additionally, all existing callers > pass in a constant max_longs parameter. Taken together, these two things > mean that the codegen doesn't really change much for one-word-at-a-time > platforms, while performance is greatly improved on platforms such as > s390. > > Cc: Will Deacon <will@kernel.org> > Cc: Alexander Gordeev <agordeev@linux.ibm.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: H. Peter Anvin <hpa@zytor.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Johannes Berg <johannes@sipsolutions.net> > Cc: Mark Rutland <mark.rutland@arm.com> > Cc: Harald Freudenberger <freude@linux.ibm.com> > Acked-by: Michael Ellerman <mpe@ellerman.id.au> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> > --- > arch/arm64/include/asm/archrandom.h | 102 ++++++++++++-------------- > arch/arm64/kernel/kaslr.c | 2 +- > arch/powerpc/include/asm/archrandom.h | 30 ++------ > arch/powerpc/kvm/book3s_hv.c | 2 +- > arch/s390/include/asm/archrandom.h | 29 ++------ > arch/um/include/asm/archrandom.h | 21 ++---- > arch/x86/include/asm/archrandom.h | 41 +---------- > arch/x86/kernel/espfix_64.c | 2 +- > drivers/char/random.c | 45 ++++++++---- > include/asm-generic/archrandom.h | 18 +---- > include/linux/random.h | 12 +-- > 11 files changed, 116 insertions(+), 188 deletions(-) For s390: Acked-by: Heiko Carstens <hca@linux.ibm.com> ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v3] random: handle archrandom with multiple longs 2022-07-19 13:02 ` [PATCH v3] random: handle archrandom with multiple longs Jason A. Donenfeld ` (2 preceding siblings ...) 2022-07-22 12:06 ` Heiko Carstens @ 2022-07-24 22:47 ` Jason A. Donenfeld 2022-07-25 9:19 ` Borislav Petkov ` (2 subsequent siblings) 6 siblings, 0 replies; 27+ messages in thread From: Jason A. Donenfeld @ 2022-07-24 22:47 UTC (permalink / raw) To: Borislav Petkov; +Cc: LKML, X86 ML Hey Borislav (or other x86ers), On Tue, Jul 19, 2022 at 3:02 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote: > > The archrandom interface was originally designed for x86, which supplies > RDRAND/RDSEED for receiving random words into registers, resulting in > one function to generate an int and another to generate a long. However, > other architectures don't follow this. > > On arm64, the SMCCC TRNG interface can return between 1 and 3 longs. On > s390, the CPACF TRNG interface can return arbitrary amounts, with 32 > longs having the same cost as one. On UML, the os_getrandom() interface > can return arbitrary amounts. > > So change the api signature to take a "max_longs" parameter designating > the maximum number of longs requested, and then return the number of > longs generated. > > Since callers need to check this return value and loop anyway, each arch > implementation does not bother implementing its own loop to try again to > fill the maximum number of longs. Additionally, all existing callers > pass in a constant max_longs parameter. Taken together, these two things > mean that the codegen doesn't really change much for one-word-at-a-time > platforms, while performance is greatly improved on platforms such as > s390. This patch now has acks from Heiko, Catalin, Mark, and Michael, covering the arm, ppc, and s390 changes. The changes to s390 and arm were non-trivial so those acks were quite meaningful. On x86 and ppc, the code compiles down to basically the same assembly and the change really doesn't matter at all. Nonetheless, I thought I should give you a final poke in case you want to ack or nack this, lest I step on the tip.git of your shoes. Jason ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v3] random: handle archrandom with multiple longs 2022-07-19 13:02 ` [PATCH v3] random: handle archrandom with multiple longs Jason A. Donenfeld ` (3 preceding siblings ...) 2022-07-24 22:47 ` Jason A. Donenfeld @ 2022-07-25 9:19 ` Borislav Petkov 2022-07-25 9:26 ` Jason A. Donenfeld 2022-07-25 11:25 ` [PATCH v3] random: handle archrandom with multiple longs Borislav Petkov 2022-08-01 14:46 ` Harald Freudenberger 6 siblings, 1 reply; 27+ messages in thread From: Borislav Petkov @ 2022-07-25 9:19 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Heiko Carstens, Johannes Berg, Mark Rutland, Harald Freudenberger, Michael Ellerman On Tue, Jul 19, 2022 at 03:02:07PM +0200, Jason A. Donenfeld wrote: > Since callers need to check this return value and loop anyway, each arch > implementation does not bother implementing its own loop to try again to > fill the maximum number of longs. Additionally, all existing callers > pass in a constant max_longs parameter. Hmm, maybe this has come up already but it reads weird. If I have a function arch_get_random_longs(), I'd expect it to give me the number of longs I requested or say, error. Why do the callers need to loop? If I have to loop, I'd call the "get me one long" function and loop N times. -- Regards/Gruss, Boris. SUSE Software Solutions Germany GmbH GF: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman (HRB 36809, AG Nürnberg) ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v3] random: handle archrandom with multiple longs 2022-07-25 9:19 ` Borislav Petkov @ 2022-07-25 9:26 ` Jason A. Donenfeld 2022-07-25 9:36 ` David Laight 2022-07-25 11:10 ` [PATCH] random: discourage use of archrandom outside of rng Jason A. Donenfeld 0 siblings, 2 replies; 27+ messages in thread From: Jason A. Donenfeld @ 2022-07-25 9:26 UTC (permalink / raw) To: Borislav Petkov Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Heiko Carstens, Johannes Berg, Mark Rutland, Harald Freudenberger, Michael Ellerman Hi Boris, On Mon, Jul 25, 2022 at 11:19:01AM +0200, Borislav Petkov wrote: > On Tue, Jul 19, 2022 at 03:02:07PM +0200, Jason A. Donenfeld wrote: > > Since callers need to check this return value and loop anyway, each arch > > implementation does not bother implementing its own loop to try again to > > fill the maximum number of longs. Additionally, all existing callers > > pass in a constant max_longs parameter. > > Hmm, maybe this has come up already but it reads weird. > > If I have a function arch_get_random_longs(), I'd expect it to give me > the number of longs I requested or say, error. > > Why do the callers need to loop? > > If I have to loop, I'd call the "get me one long" function and loop N > times. Answered partially in the commit message you quoted and partially here: https://lore.kernel.org/lkml/YtqIbrds53EuyqPE@zx2c4.com/ Note that arch_get_random_longs() is not a general purpose function. For that there used to be get_random_bytes_arch(), but that no longer exists as people shouldn't be using this stuff directly. arch_get_random_longs() is a special purpose function mainly intended for use by the RNG itself. More directly, the reason we don't want to error is because the use case has fallbacks meant to handle errors. The cascade looks like this (quoting from the other email): unsigned long array[whatever]; for (i = 0; i < ARRAY_SIZE(array);) { longs = arch_get_random_seed_longs(&array[i], ARRAY_SIZE(array) - i); if (longs) { i += longs; continue; } longs = arch_get_random_longs(&array[i], ARRAY_SIZE(array) - i); if (longs) { i += longs; continue; } array[i++] = random_get_entropy(); } It tries to get the best that it can as much as it can, but isn't going to block or do anything too nuts for that. Anyway, from an x86 perspective, I can't imagine you object to this change, right? Codegen is the same. Jason ^ permalink raw reply [flat|nested] 27+ messages in thread
* RE: [PATCH v3] random: handle archrandom with multiple longs 2022-07-25 9:26 ` Jason A. Donenfeld @ 2022-07-25 9:36 ` David Laight 2022-07-25 9:37 ` Jason A. Donenfeld 2022-07-25 11:10 ` [PATCH] random: discourage use of archrandom outside of rng Jason A. Donenfeld 1 sibling, 1 reply; 27+ messages in thread From: David Laight @ 2022-07-25 9:36 UTC (permalink / raw) To: 'Jason A. Donenfeld', Borislav Petkov Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Heiko Carstens, Johannes Berg, Mark Rutland, Harald Freudenberger, Michael Ellerman ... > More directly, the reason we don't want to error is because the use case > has fallbacks meant to handle errors. The cascade looks like this > (quoting from the other email): > > unsigned long array[whatever]; > for (i = 0; i < ARRAY_SIZE(array);) { > longs = arch_get_random_seed_longs(&array[i], ARRAY_SIZE(array) - i); > if (longs) { > i += longs; > continue; > } > longs = arch_get_random_longs(&array[i], ARRAY_SIZE(array) - i); > if (longs) { > i += longs; > continue; > } > array[i++] = random_get_entropy(); > } > > It tries to get the best that it can as much as it can, but isn't going > to block or do anything too nuts for that. Do you really want to retry the earlier calls that returned no data? David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales) ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v3] random: handle archrandom with multiple longs 2022-07-25 9:36 ` David Laight @ 2022-07-25 9:37 ` Jason A. Donenfeld 0 siblings, 0 replies; 27+ messages in thread From: Jason A. Donenfeld @ 2022-07-25 9:37 UTC (permalink / raw) To: David Laight Cc: Borislav Petkov, linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Heiko Carstens, Johannes Berg, Mark Rutland, Harald Freudenberger, Michael Ellerman On Mon, Jul 25, 2022 at 11:36 AM David Laight <David.Laight@aculab.com> wrote: > > ... > > More directly, the reason we don't want to error is because the use case > > has fallbacks meant to handle errors. The cascade looks like this > > (quoting from the other email): > > > > unsigned long array[whatever]; > > for (i = 0; i < ARRAY_SIZE(array);) { > > longs = arch_get_random_seed_longs(&array[i], ARRAY_SIZE(array) - i); > > if (longs) { > > i += longs; > > continue; > > } > > longs = arch_get_random_longs(&array[i], ARRAY_SIZE(array) - i); > > if (longs) { > > i += longs; > > continue; > > } > > array[i++] = random_get_entropy(); > > } > > > > It tries to get the best that it can as much as it can, but isn't going > > to block or do anything too nuts for that. > > Do you really want to retry the earlier calls that returned no data? Does the above code do that? ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH] random: discourage use of archrandom outside of rng 2022-07-25 9:26 ` Jason A. Donenfeld 2022-07-25 9:36 ` David Laight @ 2022-07-25 11:10 ` Jason A. Donenfeld 2022-07-25 11:25 ` Borislav Petkov ` (2 more replies) 1 sibling, 3 replies; 27+ messages in thread From: Jason A. Donenfeld @ 2022-07-25 11:10 UTC (permalink / raw) To: linux-kernel Cc: Jason A. Donenfeld, Borislav Petkov, Heiko Carstens, Catalin Marinas, Mark Rutland, Michael Ellerman, Johannes Berg Borislav pointed out during the review of "random: handle archrandom with multiple longs" that people might actually use this function, which might not be good because the function has surprising semantics. This of course was also the case before that patch, and indeed RDSEED-like functions across architectures often behave surprisingly, failing often. While random.c has been written specifically to work with that behavior, not much else is well equipped for that. So add a comment suggesting that this is not for general consumption. Fortunately, nobody uses this for general consumption anyway, and people who try quickly find themselves in trouble. But adding this comment out of an abundance of caution was nonetheless suggested, and it at least means there will be easier justification for cleaning up potential misuses of the function later. Cc: Borislav Petkov <bp@suse.de> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Johannes Berg <johannes@sipsolutions.net> Suggested-by: Borislav Petkov <bp@suse.de> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> --- arch/arm64/include/asm/archrandom.h | 13 +++++++++++++ arch/powerpc/include/asm/archrandom.h | 8 ++++++++ arch/s390/include/asm/archrandom.h | 8 ++++++++ arch/um/include/asm/archrandom.h | 8 ++++++++ arch/x86/include/asm/archrandom.h | 9 ++++++--- include/asm-generic/archrandom.h | 8 ++++++++ 6 files changed, 51 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h index 109e2a4454be..0b5ee0e12a13 100644 --- a/arch/arm64/include/asm/archrandom.h +++ b/arch/arm64/include/asm/archrandom.h @@ -58,6 +58,10 @@ static inline bool __arm64_rndrrs(unsigned long *v) return ok; } +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { /* @@ -71,6 +75,10 @@ static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t return 0; } +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { if (!max_longs) @@ -121,6 +129,11 @@ static inline bool __init __early_cpu_has_rndr(void) return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; } + +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __init __must_check arch_get_random_seed_longs_early(unsigned long *v, size_t max_longs) { diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h index 0e365c5b2396..7accfe346d49 100644 --- a/arch/powerpc/include/asm/archrandom.h +++ b/arch/powerpc/include/asm/archrandom.h @@ -4,11 +4,19 @@ #include <asm/machdep.h> +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { return 0; } +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { if (max_longs && ppc_md.get_random_seed && ppc_md.get_random_seed(v)) diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h index cf5e000df0a1..ae1efdd6f3a9 100644 --- a/arch/s390/include/asm/archrandom.h +++ b/arch/s390/include/asm/archrandom.h @@ -18,11 +18,19 @@ DECLARE_STATIC_KEY_FALSE(s390_arch_random_available); extern atomic64_t s390_arch_random_counter; +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { return 0; } +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { if (static_branch_likely(&s390_arch_random_available)) { diff --git a/arch/um/include/asm/archrandom.h b/arch/um/include/asm/archrandom.h index 24e16c979c51..d2b20bb0ed53 100644 --- a/arch/um/include/asm/archrandom.h +++ b/arch/um/include/asm/archrandom.h @@ -7,6 +7,10 @@ /* This is from <os.h>, but better not to #include that in a global header here. */ ssize_t os_getrandom(void *buf, size_t len, unsigned int flags); +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { ssize_t ret; @@ -17,6 +21,10 @@ static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t return ret / sizeof(*v); } +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { return 0; diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h index 02bae8e0758b..8352948e6412 100644 --- a/arch/x86/include/asm/archrandom.h +++ b/arch/x86/include/asm/archrandom.h @@ -41,15 +41,18 @@ static inline bool __must_check rdseed_long(unsigned long *v) } /* - * These are the generic interfaces; they must not be declared if the - * stubs in <linux/random.h> are to be invoked. + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. */ - static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { return max_longs && static_cpu_has(X86_FEATURE_RDRAND) && rdrand_long(v) ? 1 : 0; } +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { return max_longs && static_cpu_has(X86_FEATURE_RDSEED) && rdseed_long(v) ? 1 : 0; diff --git a/include/asm-generic/archrandom.h b/include/asm-generic/archrandom.h index 3cd7f980cfdc..800b41639dd7 100644 --- a/include/asm-generic/archrandom.h +++ b/include/asm-generic/archrandom.h @@ -2,11 +2,19 @@ #ifndef __ASM_GENERIC_ARCHRANDOM_H__ #define __ASM_GENERIC_ARCHRANDOM_H__ +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) { return 0; } +/* + * This should only be used by drivers/char/random.c. Other drivers *must* + * use get_random_bytes() instead. + */ static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) { return 0; -- 2.35.1 ^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH] random: discourage use of archrandom outside of rng 2022-07-25 11:10 ` [PATCH] random: discourage use of archrandom outside of rng Jason A. Donenfeld @ 2022-07-25 11:25 ` Borislav Petkov 2022-07-25 11:33 ` Heiko Carstens 2022-07-25 13:02 ` Mark Rutland 2 siblings, 0 replies; 27+ messages in thread From: Borislav Petkov @ 2022-07-25 11:25 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, Heiko Carstens, Catalin Marinas, Mark Rutland, Michael Ellerman, Johannes Berg On Mon, Jul 25, 2022 at 01:10:38PM +0200, Jason A. Donenfeld wrote: > Borislav pointed out during the review of "random: handle archrandom > with multiple longs" that people might actually use this function, which > might not be good because the function has surprising semantics. This of > course was also the case before that patch, and indeed RDSEED-like > functions across architectures often behave surprisingly, failing often. > While random.c has been written specifically to work with that behavior, > not much else is well equipped for that. > > So add a comment suggesting that this is not for general consumption. > Fortunately, nobody uses this for general consumption anyway, and people > who try quickly find themselves in trouble. But adding this comment out > of an abundance of caution was nonetheless suggested, and it at least > means there will be easier justification for cleaning up potential > misuses of the function later. > > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Mark Rutland <mark.rutland@arm.com> > Cc: Michael Ellerman <mpe@ellerman.id.au> > Cc: Johannes Berg <johannes@sipsolutions.net> > Suggested-by: Borislav Petkov <bp@suse.de> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> > --- > arch/arm64/include/asm/archrandom.h | 13 +++++++++++++ > arch/powerpc/include/asm/archrandom.h | 8 ++++++++ > arch/s390/include/asm/archrandom.h | 8 ++++++++ > arch/um/include/asm/archrandom.h | 8 ++++++++ > arch/x86/include/asm/archrandom.h | 9 ++++++--- > include/asm-generic/archrandom.h | 8 ++++++++ > 6 files changed, 51 insertions(+), 3 deletions(-) Acked-by: Borislav Petkov <bp@suse.de> Thx. -- Regards/Gruss, Boris. SUSE Software Solutions Germany GmbH GF: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman (HRB 36809, AG Nürnberg) ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH] random: discourage use of archrandom outside of rng 2022-07-25 11:10 ` [PATCH] random: discourage use of archrandom outside of rng Jason A. Donenfeld 2022-07-25 11:25 ` Borislav Petkov @ 2022-07-25 11:33 ` Heiko Carstens 2022-07-25 13:02 ` Mark Rutland 2 siblings, 0 replies; 27+ messages in thread From: Heiko Carstens @ 2022-07-25 11:33 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, Borislav Petkov, Catalin Marinas, Mark Rutland, Michael Ellerman, Johannes Berg On Mon, Jul 25, 2022 at 01:10:38PM +0200, Jason A. Donenfeld wrote: > Borislav pointed out during the review of "random: handle archrandom > with multiple longs" that people might actually use this function, which > might not be good because the function has surprising semantics. This of > course was also the case before that patch, and indeed RDSEED-like > functions across architectures often behave surprisingly, failing often. > While random.c has been written specifically to work with that behavior, > not much else is well equipped for that. > > So add a comment suggesting that this is not for general consumption. > Fortunately, nobody uses this for general consumption anyway, and people > who try quickly find themselves in trouble. But adding this comment out > of an abundance of caution was nonetheless suggested, and it at least > means there will be easier justification for cleaning up potential > misuses of the function later. > > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Mark Rutland <mark.rutland@arm.com> > Cc: Michael Ellerman <mpe@ellerman.id.au> > Cc: Johannes Berg <johannes@sipsolutions.net> > Suggested-by: Borislav Petkov <bp@suse.de> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> > --- > arch/arm64/include/asm/archrandom.h | 13 +++++++++++++ > arch/powerpc/include/asm/archrandom.h | 8 ++++++++ > arch/s390/include/asm/archrandom.h | 8 ++++++++ > arch/um/include/asm/archrandom.h | 8 ++++++++ > arch/x86/include/asm/archrandom.h | 9 ++++++--- > include/asm-generic/archrandom.h | 8 ++++++++ > 6 files changed, 51 insertions(+), 3 deletions(-) For s390: Acked-by: Heiko Carstens <hca@linux.ibm.com> ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH] random: discourage use of archrandom outside of rng 2022-07-25 11:10 ` [PATCH] random: discourage use of archrandom outside of rng Jason A. Donenfeld 2022-07-25 11:25 ` Borislav Petkov 2022-07-25 11:33 ` Heiko Carstens @ 2022-07-25 13:02 ` Mark Rutland 2 siblings, 0 replies; 27+ messages in thread From: Mark Rutland @ 2022-07-25 13:02 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, Borislav Petkov, Heiko Carstens, Catalin Marinas, Michael Ellerman, Johannes Berg On Mon, Jul 25, 2022 at 01:10:38PM +0200, Jason A. Donenfeld wrote: > Borislav pointed out during the review of "random: handle archrandom > with multiple longs" that people might actually use this function, which > might not be good because the function has surprising semantics. This of > course was also the case before that patch, and indeed RDSEED-like > functions across architectures often behave surprisingly, failing often. > While random.c has been written specifically to work with that behavior, > not much else is well equipped for that. > > So add a comment suggesting that this is not for general consumption. > Fortunately, nobody uses this for general consumption anyway, and people > who try quickly find themselves in trouble. But adding this comment out > of an abundance of caution was nonetheless suggested, and it at least > means there will be easier justification for cleaning up potential > misuses of the function later. > > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Mark Rutland <mark.rutland@arm.com> > Cc: Michael Ellerman <mpe@ellerman.id.au> > Cc: Johannes Berg <johannes@sipsolutions.net> > Suggested-by: Borislav Petkov <bp@suse.de> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> FWIW: Acked-by: Mark Rutland <mark.rutland@arm.com> Mark. > --- > arch/arm64/include/asm/archrandom.h | 13 +++++++++++++ > arch/powerpc/include/asm/archrandom.h | 8 ++++++++ > arch/s390/include/asm/archrandom.h | 8 ++++++++ > arch/um/include/asm/archrandom.h | 8 ++++++++ > arch/x86/include/asm/archrandom.h | 9 ++++++--- > include/asm-generic/archrandom.h | 8 ++++++++ > 6 files changed, 51 insertions(+), 3 deletions(-) > > diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h > index 109e2a4454be..0b5ee0e12a13 100644 > --- a/arch/arm64/include/asm/archrandom.h > +++ b/arch/arm64/include/asm/archrandom.h > @@ -58,6 +58,10 @@ static inline bool __arm64_rndrrs(unsigned long *v) > return ok; > } > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > /* > @@ -71,6 +75,10 @@ static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t > return 0; > } > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > if (!max_longs) > @@ -121,6 +129,11 @@ static inline bool __init __early_cpu_has_rndr(void) > return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; > } > > + > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __init __must_check > arch_get_random_seed_longs_early(unsigned long *v, size_t max_longs) > { > diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h > index 0e365c5b2396..7accfe346d49 100644 > --- a/arch/powerpc/include/asm/archrandom.h > +++ b/arch/powerpc/include/asm/archrandom.h > @@ -4,11 +4,19 @@ > > #include <asm/machdep.h> > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > return 0; > } > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > if (max_longs && ppc_md.get_random_seed && ppc_md.get_random_seed(v)) > diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h > index cf5e000df0a1..ae1efdd6f3a9 100644 > --- a/arch/s390/include/asm/archrandom.h > +++ b/arch/s390/include/asm/archrandom.h > @@ -18,11 +18,19 @@ > DECLARE_STATIC_KEY_FALSE(s390_arch_random_available); > extern atomic64_t s390_arch_random_counter; > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > return 0; > } > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > if (static_branch_likely(&s390_arch_random_available)) { > diff --git a/arch/um/include/asm/archrandom.h b/arch/um/include/asm/archrandom.h > index 24e16c979c51..d2b20bb0ed53 100644 > --- a/arch/um/include/asm/archrandom.h > +++ b/arch/um/include/asm/archrandom.h > @@ -7,6 +7,10 @@ > /* This is from <os.h>, but better not to #include that in a global header here. */ > ssize_t os_getrandom(void *buf, size_t len, unsigned int flags); > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > ssize_t ret; > @@ -17,6 +21,10 @@ static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t > return ret / sizeof(*v); > } > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > return 0; > diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h > index 02bae8e0758b..8352948e6412 100644 > --- a/arch/x86/include/asm/archrandom.h > +++ b/arch/x86/include/asm/archrandom.h > @@ -41,15 +41,18 @@ static inline bool __must_check rdseed_long(unsigned long *v) > } > > /* > - * These are the generic interfaces; they must not be declared if the > - * stubs in <linux/random.h> are to be invoked. > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > */ > - > static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > return max_longs && static_cpu_has(X86_FEATURE_RDRAND) && rdrand_long(v) ? 1 : 0; > } > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > return max_longs && static_cpu_has(X86_FEATURE_RDSEED) && rdseed_long(v) ? 1 : 0; > diff --git a/include/asm-generic/archrandom.h b/include/asm-generic/archrandom.h > index 3cd7f980cfdc..800b41639dd7 100644 > --- a/include/asm-generic/archrandom.h > +++ b/include/asm-generic/archrandom.h > @@ -2,11 +2,19 @@ > #ifndef __ASM_GENERIC_ARCHRANDOM_H__ > #define __ASM_GENERIC_ARCHRANDOM_H__ > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_longs(unsigned long *v, size_t max_longs) > { > return 0; > } > > +/* > + * This should only be used by drivers/char/random.c. Other drivers *must* > + * use get_random_bytes() instead. > + */ > static inline size_t __must_check arch_get_random_seed_longs(unsigned long *v, size_t max_longs) > { > return 0; > -- > 2.35.1 > ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v3] random: handle archrandom with multiple longs 2022-07-19 13:02 ` [PATCH v3] random: handle archrandom with multiple longs Jason A. Donenfeld ` (4 preceding siblings ...) 2022-07-25 9:19 ` Borislav Petkov @ 2022-07-25 11:25 ` Borislav Petkov 2022-08-01 14:46 ` Harald Freudenberger 6 siblings, 0 replies; 27+ messages in thread From: Borislav Petkov @ 2022-07-25 11:25 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Heiko Carstens, Johannes Berg, Mark Rutland, Harald Freudenberger, Michael Ellerman On Tue, Jul 19, 2022 at 03:02:07PM +0200, Jason A. Donenfeld wrote: > The archrandom interface was originally designed for x86, which supplies > RDRAND/RDSEED for receiving random words into registers, resulting in > one function to generate an int and another to generate a long. However, > other architectures don't follow this. > > On arm64, the SMCCC TRNG interface can return between 1 and 3 longs. On > s390, the CPACF TRNG interface can return arbitrary amounts, with 32 > longs having the same cost as one. On UML, the os_getrandom() interface > can return arbitrary amounts. > > So change the api signature to take a "max_longs" parameter designating > the maximum number of longs requested, and then return the number of > longs generated. > > Since callers need to check this return value and loop anyway, each arch > implementation does not bother implementing its own loop to try again to > fill the maximum number of longs. Additionally, all existing callers > pass in a constant max_longs parameter. Taken together, these two things > mean that the codegen doesn't really change much for one-word-at-a-time > platforms, while performance is greatly improved on platforms such as > s390. > > Cc: Will Deacon <will@kernel.org> > Cc: Alexander Gordeev <agordeev@linux.ibm.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: H. Peter Anvin <hpa@zytor.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Johannes Berg <johannes@sipsolutions.net> > Cc: Mark Rutland <mark.rutland@arm.com> > Cc: Harald Freudenberger <freude@linux.ibm.com> > Acked-by: Michael Ellerman <mpe@ellerman.id.au> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> > --- > arch/arm64/include/asm/archrandom.h | 102 ++++++++++++-------------- > arch/arm64/kernel/kaslr.c | 2 +- > arch/powerpc/include/asm/archrandom.h | 30 ++------ > arch/powerpc/kvm/book3s_hv.c | 2 +- > arch/s390/include/asm/archrandom.h | 29 ++------ > arch/um/include/asm/archrandom.h | 21 ++---- > arch/x86/include/asm/archrandom.h | 41 +---------- > arch/x86/kernel/espfix_64.c | 2 +- > drivers/char/random.c | 45 ++++++++---- > include/asm-generic/archrandom.h | 18 +---- > include/linux/random.h | 12 +-- > 11 files changed, 116 insertions(+), 188 deletions(-) Acked-by: Borislav Petkov <bp@suse.de> # for x86 Thx. -- Regards/Gruss, Boris. SUSE Software Solutions Germany GmbH GF: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman (HRB 36809, AG Nürnberg) ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v3] random: handle archrandom with multiple longs 2022-07-19 13:02 ` [PATCH v3] random: handle archrandom with multiple longs Jason A. Donenfeld ` (5 preceding siblings ...) 2022-07-25 11:25 ` [PATCH v3] random: handle archrandom with multiple longs Borislav Petkov @ 2022-08-01 14:46 ` Harald Freudenberger 2022-08-01 14:50 ` Jason A. Donenfeld 6 siblings, 1 reply; 27+ messages in thread From: Harald Freudenberger @ 2022-08-01 14:46 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Mark Rutland, Michael Ellerman On 2022-07-19 15:02, Jason A. Donenfeld wrote: > The archrandom interface was originally designed for x86, which > supplies > RDRAND/RDSEED for receiving random words into registers, resulting in > one function to generate an int and another to generate a long. > However, > other architectures don't follow this. > > On arm64, the SMCCC TRNG interface can return between 1 and 3 longs. On > s390, the CPACF TRNG interface can return arbitrary amounts, with 32 > longs having the same cost as one. On UML, the os_getrandom() interface > can return arbitrary amounts. > > So change the api signature to take a "max_longs" parameter designating > the maximum number of longs requested, and then return the number of > longs generated. > > Since callers need to check this return value and loop anyway, each > arch > implementation does not bother implementing its own loop to try again > to > fill the maximum number of longs. Additionally, all existing callers > pass in a constant max_longs parameter. Taken together, these two > things > mean that the codegen doesn't really change much for one-word-at-a-time > platforms, while performance is greatly improved on platforms such as > s390. > > Cc: Will Deacon <will@kernel.org> > Cc: Alexander Gordeev <agordeev@linux.ibm.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: H. Peter Anvin <hpa@zytor.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Johannes Berg <johannes@sipsolutions.net> > Cc: Mark Rutland <mark.rutland@arm.com> > Cc: Harald Freudenberger <freude@linux.ibm.com> > Acked-by: Michael Ellerman <mpe@ellerman.id.au> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> > --- > arch/arm64/include/asm/archrandom.h | 102 ++++++++++++-------------- > arch/arm64/kernel/kaslr.c | 2 +- > arch/powerpc/include/asm/archrandom.h | 30 ++------ > arch/powerpc/kvm/book3s_hv.c | 2 +- > arch/s390/include/asm/archrandom.h | 29 ++------ > arch/um/include/asm/archrandom.h | 21 ++---- > arch/x86/include/asm/archrandom.h | 41 +---------- > arch/x86/kernel/espfix_64.c | 2 +- > drivers/char/random.c | 45 ++++++++---- > include/asm-generic/archrandom.h | 18 +---- > include/linux/random.h | 12 +-- > 11 files changed, 116 insertions(+), 188 deletions(-) > > diff --git a/arch/arm64/include/asm/archrandom.h > b/arch/arm64/include/asm/archrandom.h > index c3b9fa56af67..109e2a4454be 100644 > --- a/arch/arm64/include/asm/archrandom.h > +++ b/arch/arm64/include/asm/archrandom.h > @@ -58,7 +58,7 @@ static inline bool __arm64_rndrrs(unsigned long *v) > return ok; > } > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long > *v, size_t max_longs) > { > /* > * Only support the generic interface after we have detected > @@ -66,27 +66,15 @@ static inline bool __must_check > arch_get_random_long(unsigned long *v) > * cpufeature code and with potential scheduling between CPUs > * with and without the feature. > */ > - if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) > - return true; > - return false; > + if (max_longs && cpus_have_const_cap(ARM64_HAS_RNG) && > __arm64_rndr(v)) > + return 1; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned > long *v, size_t max_longs) > { > - if (cpus_have_const_cap(ARM64_HAS_RNG)) { > - unsigned long val; > - > - if (__arm64_rndr(&val)) { > - *v = val; > - return true; > - } > - } > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned > long *v) > -{ > - struct arm_smccc_res res; > + if (!max_longs) > + return 0; > > /* > * We prefer the SMCCC call, since its semantics (return actual > @@ -95,10 +83,23 @@ static inline bool __must_check > arch_get_random_seed_long(unsigned long *v) > * (the output of a pseudo RNG freshly seeded by a TRNG). > */ > if (smccc_trng_available) { > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); > + struct arm_smccc_res res; > + > + max_longs = min_t(size_t, 3, max_longs); > + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, max_longs * 64, &res); > if ((int)res.a0 >= 0) { > - *v = res.a3; > - return true; > + switch (max_longs) { > + case 3: > + *v++ = res.a1; > + fallthrough; > + case 2: > + *v++ = res.a2; > + fallthrough; > + case 1: > + *v++ = res.a3; > + break; > + } > + return max_longs; > } > } > > @@ -108,32 +109,9 @@ static inline bool __must_check > arch_get_random_seed_long(unsigned long *v) > * enough to implement this API if no other entropy source exists. > */ > if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndrrs(v)) > - return true; > + return 1; > > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int > *v) > -{ > - struct arm_smccc_res res; > - unsigned long val; > - > - if (smccc_trng_available) { > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 32, &res); > - if ((int)res.a0 >= 0) { > - *v = res.a3 & GENMASK(31, 0); > - return true; > - } > - } > - > - if (cpus_have_const_cap(ARM64_HAS_RNG)) { > - if (__arm64_rndrrs(&val)) { > - *v = val; > - return true; > - } > - } > - > - return false; > + return 0; > } > > static inline bool __init __early_cpu_has_rndr(void) > @@ -143,26 +121,40 @@ static inline bool __init > __early_cpu_has_rndr(void) > return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; > } > > -static inline bool __init __must_check > -arch_get_random_seed_long_early(unsigned long *v) > +static inline size_t __init __must_check > +arch_get_random_seed_longs_early(unsigned long *v, size_t max_longs) > { > WARN_ON(system_state != SYSTEM_BOOTING); > > + if (!max_longs) > + return 0; > + > if (smccc_trng_available) { > struct arm_smccc_res res; > > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); > + max_longs = min_t(size_t, 3, max_longs); > + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, max_longs * 64, &res); > if ((int)res.a0 >= 0) { > - *v = res.a3; > - return true; > + switch (max_longs) { > + case 3: > + *v++ = res.a1; > + fallthrough; > + case 2: > + *v++ = res.a2; > + fallthrough; > + case 1: > + *v++ = res.a3; > + break; > + } > + return max_longs; > } > } > > if (__early_cpu_has_rndr() && __arm64_rndr(v)) > - return true; > + return 1; > > - return false; > + return 0; > } > -#define arch_get_random_seed_long_early > arch_get_random_seed_long_early > +#define arch_get_random_seed_longs_early > arch_get_random_seed_longs_early > > #endif /* _ASM_ARCHRANDOM_H */ > diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c > index 418b2bba1521..c5d541f358d3 100644 > --- a/arch/arm64/kernel/kaslr.c > +++ b/arch/arm64/kernel/kaslr.c > @@ -106,7 +106,7 @@ u64 __init kaslr_early_init(void) > * and supported. > */ > > - if (arch_get_random_seed_long_early(&raw)) > + if (arch_get_random_seed_longs_early(&raw, 1)) > seed ^= raw; > > if (!seed) { > diff --git a/arch/powerpc/include/asm/archrandom.h > b/arch/powerpc/include/asm/archrandom.h > index 25ba65df6b1a..0e365c5b2396 100644 > --- a/arch/powerpc/include/asm/archrandom.h > +++ b/arch/powerpc/include/asm/archrandom.h > @@ -4,34 +4,16 @@ > > #include <asm/machdep.h> > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long > *v, size_t max_longs) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned > long *v, size_t max_longs) > { > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned > long *v) > -{ > - if (ppc_md.get_random_seed) > - return ppc_md.get_random_seed(v); > - > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int > *v) > -{ > - unsigned long val; > - bool rc; > - > - rc = arch_get_random_seed_long(&val); > - if (rc) > - *v = val; > - > - return rc; > + if (max_longs && ppc_md.get_random_seed && ppc_md.get_random_seed(v)) > + return 1; > + return 0; > } > > #ifdef CONFIG_PPC_POWERNV > diff --git a/arch/powerpc/kvm/book3s_hv.c > b/arch/powerpc/kvm/book3s_hv.c > index e08fb3124dca..631062cde6b4 100644 > --- a/arch/powerpc/kvm/book3s_hv.c > +++ b/arch/powerpc/kvm/book3s_hv.c > @@ -1207,7 +1207,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu > *vcpu) > break; > #endif > case H_RANDOM: > - if (!arch_get_random_seed_long(&vcpu->arch.regs.gpr[4])) > + if (!arch_get_random_seed_longs(&vcpu->arch.regs.gpr[4], 1)) > ret = H_HARDWARE; > break; > case H_RPT_INVALIDATE: > diff --git a/arch/s390/include/asm/archrandom.h > b/arch/s390/include/asm/archrandom.h > index 0a1c2e66c709..cf5e000df0a1 100644 > --- a/arch/s390/include/asm/archrandom.h > +++ b/arch/s390/include/asm/archrandom.h > @@ -18,34 +18,19 @@ > DECLARE_STATIC_KEY_FALSE(s390_arch_random_available); > extern atomic64_t s390_arch_random_counter; > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long > *v, size_t max_longs) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned > long *v) > -{ > - if (static_branch_likely(&s390_arch_random_available)) { > - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); > - atomic64_add(sizeof(*v), &s390_arch_random_counter); > - return true; > - } > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int > *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned > long *v, size_t max_longs) > { > if (static_branch_likely(&s390_arch_random_available)) { > - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); > - atomic64_add(sizeof(*v), &s390_arch_random_counter); > - return true; > + cpacf_trng(NULL, 0, (u8 *)v, max_longs * sizeof(*v)); > + atomic64_add(max_longs * sizeof(*v), &s390_arch_random_counter); > + return max_longs; > } > - return false; > + return 0; > } > > #endif /* _ASM_S390_ARCHRANDOM_H */ > diff --git a/arch/um/include/asm/archrandom.h > b/arch/um/include/asm/archrandom.h > index 2f24cb96391d..24e16c979c51 100644 > --- a/arch/um/include/asm/archrandom.h > +++ b/arch/um/include/asm/archrandom.h > @@ -7,24 +7,19 @@ > /* This is from <os.h>, but better not to #include that in a global > header here. */ > ssize_t os_getrandom(void *buf, size_t len, unsigned int flags); > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long > *v, size_t max_longs) > { > - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); > -} > + ssize_t ret; > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned > long *v) > -{ > - return false; > + ret = os_getrandom(v, max_longs * sizeof(*v), 0); > + if (ret < 0) > + return 0; > + return ret / sizeof(*v); > } > > -static inline bool __must_check arch_get_random_seed_int(unsigned int > *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned > long *v, size_t max_longs) > { > - return false; > + return 0; > } > > #endif > diff --git a/arch/x86/include/asm/archrandom.h > b/arch/x86/include/asm/archrandom.h > index fb235b696175..02bae8e0758b 100644 > --- a/arch/x86/include/asm/archrandom.h > +++ b/arch/x86/include/asm/archrandom.h > @@ -31,20 +31,6 @@ static inline bool __must_check rdrand_long(unsigned > long *v) > return false; > } > > -static inline bool __must_check rdrand_int(unsigned int *v) > -{ > - bool ok; > - unsigned int retry = RDRAND_RETRY_LOOPS; > - do { > - asm volatile("rdrand %[out]" > - CC_SET(c) > - : CC_OUT(c) (ok), [out] "=r" (*v)); > - if (ok) > - return true; > - } while (--retry); > - return false; > -} > - > static inline bool __must_check rdseed_long(unsigned long *v) > { > bool ok; > @@ -54,38 +40,19 @@ static inline bool __must_check > rdseed_long(unsigned long *v) > return ok; > } > > -static inline bool __must_check rdseed_int(unsigned int *v) > -{ > - bool ok; > - asm volatile("rdseed %[out]" > - CC_SET(c) > - : CC_OUT(c) (ok), [out] "=r" (*v)); > - return ok; > -} > - > /* > * These are the generic interfaces; they must not be declared if the > * stubs in <linux/random.h> are to be invoked. > */ > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > -{ > - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_long(v) : false; > -} > - > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_int(v) : false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned > long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long > *v, size_t max_longs) > { > - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_long(v) : false; > + return max_longs && static_cpu_has(X86_FEATURE_RDRAND) && > rdrand_long(v) ? 1 : 0; > } > > -static inline bool __must_check arch_get_random_seed_int(unsigned int > *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned > long *v, size_t max_longs) > { > - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_int(v) : false; > + return max_longs && static_cpu_has(X86_FEATURE_RDSEED) && > rdseed_long(v) ? 1 : 0; > } > > #ifndef CONFIG_UML > diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c > index 4fe7af58cfe1..9417d5aa7305 100644 > --- a/arch/x86/kernel/espfix_64.c > +++ b/arch/x86/kernel/espfix_64.c > @@ -100,7 +100,7 @@ static void init_espfix_random(void) > * This is run before the entropy pools are initialized, > * but this is hopefully better than nothing. > */ > - if (!arch_get_random_long(&rand)) { > + if (!arch_get_random_longs(&rand, 1)) { > /* The constant is an arbitrary large prime */ > rand = rdtsc(); > rand *= 0xc345c6b72fd16123UL; > diff --git a/drivers/char/random.c b/drivers/char/random.c > index 0c6568ae5f68..7bf11fa66265 100644 > --- a/drivers/char/random.c > +++ b/drivers/char/random.c > @@ -596,12 +596,20 @@ static void extract_entropy(void *buf, size_t > len) > unsigned long rdseed[32 / sizeof(long)]; > size_t counter; > } block; > - size_t i; > + size_t i, longs; > > - for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) { > - if (!arch_get_random_seed_long(&block.rdseed[i]) && > - !arch_get_random_long(&block.rdseed[i])) > - block.rdseed[i] = random_get_entropy(); > + for (i = 0; i < ARRAY_SIZE(block.rdseed);) { > + longs = arch_get_random_seed_longs(&block.rdseed[i], > ARRAY_SIZE(block.rdseed) - i); > + if (longs) { > + i += longs; > + continue; > + } > + longs = arch_get_random_longs(&block.rdseed[i], > ARRAY_SIZE(block.rdseed) - i); > + if (longs) { > + i += longs; > + continue; > + } > + block.rdseed[i++] = random_get_entropy(); > } > > spin_lock_irqsave(&input_pool.lock, flags); > @@ -776,22 +784,31 @@ static struct notifier_block pm_notifier = { > .notifier_call = random_pm_notifica > int __init random_init(const char *command_line) > { > ktime_t now = ktime_get_real(); > - unsigned int i, arch_bits; > - unsigned long entropy; > + size_t i, longs, arch_bits; > + unsigned long entropy[BLAKE2S_BLOCK_SIZE / sizeof(long)]; > > #if defined(LATENT_ENTROPY_PLUGIN) > static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst > __latent_entropy; > _mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed)); > #endif > > - for (i = 0, arch_bits = BLAKE2S_BLOCK_SIZE * 8; > - i < BLAKE2S_BLOCK_SIZE; i += sizeof(entropy)) { > - if (!arch_get_random_seed_long_early(&entropy) && > - !arch_get_random_long_early(&entropy)) { > - entropy = random_get_entropy(); > - arch_bits -= sizeof(entropy) * 8; > + for (i = 0, arch_bits = sizeof(entropy) * 8; i < > ARRAY_SIZE(entropy);) { > + longs = arch_get_random_seed_longs(entropy, ARRAY_SIZE(entropy) - > i); > + if (longs) { > + _mix_pool_bytes(entropy, sizeof(*entropy) * longs); > + i += longs; > + continue; > } > - _mix_pool_bytes(&entropy, sizeof(entropy)); > + longs = arch_get_random_longs(entropy, ARRAY_SIZE(entropy) - i); > + if (longs) { > + _mix_pool_bytes(entropy, sizeof(*entropy) * longs); > + i += longs; > + continue; > + } > + entropy[0] = random_get_entropy(); > + _mix_pool_bytes(entropy, sizeof(*entropy)); > + arch_bits -= sizeof(*entropy) * 8; > + ++i; > } > _mix_pool_bytes(&now, sizeof(now)); > _mix_pool_bytes(utsname(), sizeof(*(utsname()))); > diff --git a/include/asm-generic/archrandom.h > b/include/asm-generic/archrandom.h > index 3a5ee202dd86..3cd7f980cfdc 100644 > --- a/include/asm-generic/archrandom.h > +++ b/include/asm-generic/archrandom.h > @@ -2,24 +2,14 @@ > #ifndef __ASM_GENERIC_ARCHRANDOM_H__ > #define __ASM_GENERIC_ARCHRANDOM_H__ > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_longs(unsigned long > *v, size_t max_longs) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_longs(unsigned > long *v, size_t max_longs) > { > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned > long *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int > *v) > -{ > - return false; > + return 0; > } > > #endif > diff --git a/include/linux/random.h b/include/linux/random.h > index 865770e29f3e..3fec206487f6 100644 > --- a/include/linux/random.h > +++ b/include/linux/random.h > @@ -112,19 +112,19 @@ declare_get_random_var_wait(long, unsigned long) > * Called from the boot CPU during startup; not valid to call once > * secondary CPUs are up and preemption is possible. > */ > -#ifndef arch_get_random_seed_long_early > -static inline bool __init arch_get_random_seed_long_early(unsigned > long *v) > +#ifndef arch_get_random_seed_longs_early > +static inline size_t __init arch_get_random_seed_longs_early(unsigned > long *v, size_t max_longs) > { > WARN_ON(system_state != SYSTEM_BOOTING); > - return arch_get_random_seed_long(v); > + return arch_get_random_seed_longs(v, max_longs); > } > #endif > > -#ifndef arch_get_random_long_early > -static inline bool __init arch_get_random_long_early(unsigned long *v) > +#ifndef arch_get_random_longs_early > +static inline bool __init arch_get_random_longs_early(unsigned long > *v, size_t max_longs) > { > WARN_ON(system_state != SYSTEM_BOOTING); > - return arch_get_random_long(v); > + return arch_get_random_longs(v, max_longs); > } > #endif Nice interface :-) Here is my Acked-by: Harald Freudenberger <freude@linux.ibm.com> but please keep in mind that your patch does not apply cleanly as there is my other patch 918e75f77af7 ("s390/archrandom: prevent CPACF trng invocations in interrupt context") in between. Thanks for your work Harald ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v3] random: handle archrandom with multiple longs 2022-08-01 14:46 ` Harald Freudenberger @ 2022-08-01 14:50 ` Jason A. Donenfeld 0 siblings, 0 replies; 27+ messages in thread From: Jason A. Donenfeld @ 2022-08-01 14:50 UTC (permalink / raw) To: freude Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Mark Rutland, Michael Ellerman, Stephen Rothwell Hi Harald, On Mon, Aug 1, 2022 at 4:47 PM Harald Freudenberger <freude@linux.ibm.com> wrote: > > Nice interface :-) > Here is my > Acked-by: Harald Freudenberger <freude@linux.ibm.com> You're too late; I already sent in my pull request. But that's okay; I received Heiko's ack on this for s390. > but please keep in mind that your patch does not apply cleanly as there > is my other patch > 918e75f77af7 ("s390/archrandom: prevent CPACF trng invocations in > interrupt context") > in between. Yea, this was a late addition to 5.19, and my random tree was based on an earlier rc of 5.19. That kind of thing happens and is fine; Stephen caught it when doing the next merge and carries the merge conflict resolution there. I mentioned it to Linus in my pull, so he'll be able to work it out. Jason ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2] random: handle archrandom in plural words 2022-07-17 20:03 ` [PATCH v2] " Jason A. Donenfeld 2022-07-18 6:31 ` Michael Ellerman 2022-07-19 12:42 ` Mark Rutland @ 2022-07-22 8:08 ` Holger Dengler 2022-07-22 11:22 ` Jason A. Donenfeld 2 siblings, 1 reply; 27+ messages in thread From: Holger Dengler @ 2022-07-22 8:08 UTC (permalink / raw) To: Jason A. Donenfeld, linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86 Cc: Will Deacon, Michael Ellerman, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Harald Freudenberger Hi Jason, On 17/07/2022 22:03, Jason A. Donenfeld wrote: > The archrandom interface was originally designed for x86, which supplies > RDRAND/RDSEED for receiving random words into registers, resulting in > one function to generate an int and another to generate a long. However, > other architectures don't follow this. > > On arm64, the SMCCC TRNG interface can return between 1 and 3 words. On > s390, the CPACF TRNG interface can return between 1 and 32 words for the > same cost as for one word. On UML, the os_getrandom() interface can return > arbitrary amounts. > > So change the api signature to take a "words" parameter designating the > maximum number of words requested, and then return the number of words > generated. Why not changing the API to take bytes instead of words? Sure, at the moment it looks like all platforms with TRNG support are able to deliver at least one word, but bytes would be more flexible. > Since callers need to check this return value and loop anyway, each arch > implementation does not bother implementing its own loop to try again to > fill the requested number of words. Additionally, all existing callers > pass in a constant words parameter. Taken together, these two things > mean that the codegen doesn't really change much for one-word-at-a-time > platforms, while performance is greatly improved on platforms such as > s390. > > Cc: Will Deacon <will@kernel.org> > Cc: Michael Ellerman <mpe@ellerman.id.au> > Cc: Alexander Gordeev <agordeev@linux.ibm.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: H. Peter Anvin <hpa@zytor.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Borislav Petkov <bp@suse.de> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Johannes Berg <johannes@sipsolutions.net> > Cc: Harald Freudenberger <freude@linux.ibm.com> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> > --- > arch/arm64/include/asm/archrandom.h | 102 ++++++++++++-------------- > arch/arm64/kernel/kaslr.c | 2 +- > arch/powerpc/include/asm/archrandom.h | 30 ++------ > arch/powerpc/kvm/book3s_hv.c | 2 +- > arch/s390/include/asm/archrandom.h | 29 ++------ > arch/um/include/asm/archrandom.h | 21 ++---- > arch/x86/include/asm/archrandom.h | 41 +---------- > arch/x86/kernel/espfix_64.c | 2 +- > drivers/char/random.c | 45 ++++++++---- > include/asm-generic/archrandom.h | 18 +---- > include/linux/random.h | 12 +-- > 11 files changed, 116 insertions(+), 188 deletions(-) > > diff --git a/arch/arm64/include/asm/archrandom.h b/arch/arm64/include/asm/archrandom.h > index c3b9fa56af67..7a24fdee3e2f 100644 > --- a/arch/arm64/include/asm/archrandom.h > +++ b/arch/arm64/include/asm/archrandom.h > @@ -58,7 +58,7 @@ static inline bool __arm64_rndrrs(unsigned long *v) > return ok; > } > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > /* > * Only support the generic interface after we have detected > @@ -66,27 +66,15 @@ static inline bool __must_check arch_get_random_long(unsigned long *v) > * cpufeature code and with potential scheduling between CPUs > * with and without the feature. > */ > - if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) > - return true; > - return false; > + if (words && cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndr(v)) > + return 1; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - if (cpus_have_const_cap(ARM64_HAS_RNG)) { > - unsigned long val; > - > - if (__arm64_rndr(&val)) { > - *v = val; > - return true; > - } > - } > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - struct arm_smccc_res res; > + if (!words) > + return 0; > > /* > * We prefer the SMCCC call, since its semantics (return actual > @@ -95,10 +83,23 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > * (the output of a pseudo RNG freshly seeded by a TRNG). > */ > if (smccc_trng_available) { > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); > + struct arm_smccc_res res; > + > + words = min_t(size_t, 3, words); > + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, words * 64, &res); > if ((int)res.a0 >= 0) { > - *v = res.a3; > - return true; > + switch (words) { > + case 3: > + *v++ = res.a1; > + fallthrough; > + case 2: > + *v++ = res.a2; > + fallthrough; > + case 1: > + *v++ = res.a3; > + break; > + } > + return words; > } > } > > @@ -108,32 +109,9 @@ static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > * enough to implement this API if no other entropy source exists. > */ > if (cpus_have_const_cap(ARM64_HAS_RNG) && __arm64_rndrrs(v)) > - return true; > + return 1; > > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - struct arm_smccc_res res; > - unsigned long val; > - > - if (smccc_trng_available) { > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 32, &res); > - if ((int)res.a0 >= 0) { > - *v = res.a3 & GENMASK(31, 0); > - return true; > - } > - } > - > - if (cpus_have_const_cap(ARM64_HAS_RNG)) { > - if (__arm64_rndrrs(&val)) { > - *v = val; > - return true; > - } > - } > - > - return false; > + return 0; > } > > static inline bool __init __early_cpu_has_rndr(void) > @@ -143,26 +121,40 @@ static inline bool __init __early_cpu_has_rndr(void) > return (ftr >> ID_AA64ISAR0_EL1_RNDR_SHIFT) & 0xf; > } > > -static inline bool __init __must_check > -arch_get_random_seed_long_early(unsigned long *v) > +static inline size_t __init __must_check > +arch_get_random_seed_words_early(unsigned long *v, size_t words) > { > WARN_ON(system_state != SYSTEM_BOOTING); > > + if (!words) > + return 0; > + > if (smccc_trng_available) { > struct arm_smccc_res res; > > - arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, 64, &res); > + words = min_t(size_t, 3, words); > + arm_smccc_1_1_invoke(ARM_SMCCC_TRNG_RND64, words * 64, &res); > if ((int)res.a0 >= 0) { > - *v = res.a3; > - return true; > + switch (words) { > + case 3: > + *v++ = res.a1; > + fallthrough; > + case 2: > + *v++ = res.a2; > + fallthrough; > + case 1: > + *v++ = res.a3; > + break; > + } > + return words; > } > } > > if (__early_cpu_has_rndr() && __arm64_rndr(v)) > - return true; > + return 1; > > - return false; > + return 0; > } > -#define arch_get_random_seed_long_early arch_get_random_seed_long_early > +#define arch_get_random_seed_words_early arch_get_random_seed_words_early > > #endif /* _ASM_ARCHRANDOM_H */ > diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c > index 418b2bba1521..ed77afe16121 100644 > --- a/arch/arm64/kernel/kaslr.c > +++ b/arch/arm64/kernel/kaslr.c > @@ -106,7 +106,7 @@ u64 __init kaslr_early_init(void) > * and supported. > */ > > - if (arch_get_random_seed_long_early(&raw)) > + if (arch_get_random_seed_words_early(&raw, 1)) > seed ^= raw; > > if (!seed) { > diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h > index 25ba65df6b1a..bf2182f80480 100644 > --- a/arch/powerpc/include/asm/archrandom.h > +++ b/arch/powerpc/include/asm/archrandom.h > @@ -4,34 +4,16 @@ > > #include <asm/machdep.h> > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - if (ppc_md.get_random_seed) > - return ppc_md.get_random_seed(v); > - > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - unsigned long val; > - bool rc; > - > - rc = arch_get_random_seed_long(&val); > - if (rc) > - *v = val; > - > - return rc; > + if (words && ppc_md.get_random_seed && ppc_md.get_random_seed(v)) > + return 1; > + return 0; > } > > #ifdef CONFIG_PPC_POWERNV > diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c > index e08fb3124dca..18b2d80996b6 100644 > --- a/arch/powerpc/kvm/book3s_hv.c > +++ b/arch/powerpc/kvm/book3s_hv.c > @@ -1207,7 +1207,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) > break; > #endif > case H_RANDOM: > - if (!arch_get_random_seed_long(&vcpu->arch.regs.gpr[4])) > + if (!arch_get_random_seed_words(&vcpu->arch.regs.gpr[4], 1)) > ret = H_HARDWARE; > break; > case H_RPT_INVALIDATE: > diff --git a/arch/s390/include/asm/archrandom.h b/arch/s390/include/asm/archrandom.h > index 0a1c2e66c709..29f1a9bc3867 100644 > --- a/arch/s390/include/asm/archrandom.h > +++ b/arch/s390/include/asm/archrandom.h > @@ -18,34 +18,19 @@ > DECLARE_STATIC_KEY_FALSE(s390_arch_random_available); > extern atomic64_t s390_arch_random_counter; > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - if (static_branch_likely(&s390_arch_random_available)) { > - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); > - atomic64_add(sizeof(*v), &s390_arch_random_counter); > - return true; > - } > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > if (static_branch_likely(&s390_arch_random_available)) { > - cpacf_trng(NULL, 0, (u8 *)v, sizeof(*v)); > - atomic64_add(sizeof(*v), &s390_arch_random_counter); > - return true; > + cpacf_trng(NULL, 0, (u8 *)v, words * sizeof(*v)); > + atomic64_add(words * sizeof(*v), &s390_arch_random_counter); > + return words; > } > - return false; > + return 0; > } > > #endif /* _ASM_S390_ARCHRANDOM_H */ > diff --git a/arch/um/include/asm/archrandom.h b/arch/um/include/asm/archrandom.h > index 2f24cb96391d..6bcbd47fcb62 100644 > --- a/arch/um/include/asm/archrandom.h > +++ b/arch/um/include/asm/archrandom.h > @@ -7,24 +7,19 @@ > /* This is from <os.h>, but better not to #include that in a global header here. */ > ssize_t os_getrandom(void *buf, size_t len, unsigned int flags); > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); > -} > + ssize_t ret; > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return os_getrandom(v, sizeof(*v), 0) == sizeof(*v); > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - return false; > + ret = os_getrandom(v, words * sizeof(*v), 0); > + if (ret < 0) > + return 0; > + return ret / sizeof(*v); > } > > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return false; > + return 0; > } > > #endif > diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h > index fb235b696175..a1717b81d876 100644 > --- a/arch/x86/include/asm/archrandom.h > +++ b/arch/x86/include/asm/archrandom.h > @@ -31,20 +31,6 @@ static inline bool __must_check rdrand_long(unsigned long *v) > return false; > } > > -static inline bool __must_check rdrand_int(unsigned int *v) > -{ > - bool ok; > - unsigned int retry = RDRAND_RETRY_LOOPS; > - do { > - asm volatile("rdrand %[out]" > - CC_SET(c) > - : CC_OUT(c) (ok), [out] "=r" (*v)); > - if (ok) > - return true; > - } while (--retry); > - return false; > -} > - > static inline bool __must_check rdseed_long(unsigned long *v) > { > bool ok; > @@ -54,38 +40,19 @@ static inline bool __must_check rdseed_long(unsigned long *v) > return ok; > } > > -static inline bool __must_check rdseed_int(unsigned int *v) > -{ > - bool ok; > - asm volatile("rdseed %[out]" > - CC_SET(c) > - : CC_OUT(c) (ok), [out] "=r" (*v)); > - return ok; > -} > - > /* > * These are the generic interfaces; they must not be declared if the > * stubs in <linux/random.h> are to be invoked. > */ > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > -{ > - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_long(v) : false; > -} > - > -static inline bool __must_check arch_get_random_int(unsigned int *v) > -{ > - return static_cpu_has(X86_FEATURE_RDRAND) ? rdrand_int(v) : false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_long(v) : false; > + return words && static_cpu_has(X86_FEATURE_RDRAND) && rdrand_long(v) ? 1 : 0; > } > > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return static_cpu_has(X86_FEATURE_RDSEED) ? rdseed_int(v) : false; > + return words && static_cpu_has(X86_FEATURE_RDSEED) && rdseed_long(v) ? 1 : 0; > } > > #ifndef CONFIG_UML > diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c > index 4fe7af58cfe1..f46c9ff3c0d4 100644 > --- a/arch/x86/kernel/espfix_64.c > +++ b/arch/x86/kernel/espfix_64.c > @@ -100,7 +100,7 @@ static void init_espfix_random(void) > * This is run before the entropy pools are initialized, > * but this is hopefully better than nothing. > */ > - if (!arch_get_random_long(&rand)) { > + if (!arch_get_random_words(&rand, 1)) { > /* The constant is an arbitrary large prime */ > rand = rdtsc(); > rand *= 0xc345c6b72fd16123UL; > diff --git a/drivers/char/random.c b/drivers/char/random.c > index 0c6568ae5f68..70d8d1d7e2d7 100644 > --- a/drivers/char/random.c > +++ b/drivers/char/random.c > @@ -596,12 +596,20 @@ static void extract_entropy(void *buf, size_t len) > unsigned long rdseed[32 / sizeof(long)]; > size_t counter; > } block; > - size_t i; > + size_t i, words; > > - for (i = 0; i < ARRAY_SIZE(block.rdseed); ++i) { > - if (!arch_get_random_seed_long(&block.rdseed[i]) && > - !arch_get_random_long(&block.rdseed[i])) > - block.rdseed[i] = random_get_entropy(); > + for (i = 0; i < ARRAY_SIZE(block.rdseed);) { > + words = arch_get_random_seed_words(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); > + if (words) { > + i += words; > + continue; > + } > + words = arch_get_random_words(&block.rdseed[i], ARRAY_SIZE(block.rdseed) - i); > + if (words) { > + i += words; > + continue; > + } > + block.rdseed[i++] = random_get_entropy(); > } > > spin_lock_irqsave(&input_pool.lock, flags); > @@ -776,22 +784,31 @@ static struct notifier_block pm_notifier = { .notifier_call = random_pm_notifica > int __init random_init(const char *command_line) > { > ktime_t now = ktime_get_real(); > - unsigned int i, arch_bits; > - unsigned long entropy; > + size_t i, words, arch_bits; > + unsigned long entropy[BLAKE2S_BLOCK_SIZE / sizeof(long)]; > > #if defined(LATENT_ENTROPY_PLUGIN) > static const u8 compiletime_seed[BLAKE2S_BLOCK_SIZE] __initconst __latent_entropy; > _mix_pool_bytes(compiletime_seed, sizeof(compiletime_seed)); > #endif > > - for (i = 0, arch_bits = BLAKE2S_BLOCK_SIZE * 8; > - i < BLAKE2S_BLOCK_SIZE; i += sizeof(entropy)) { > - if (!arch_get_random_seed_long_early(&entropy) && > - !arch_get_random_long_early(&entropy)) { > - entropy = random_get_entropy(); > - arch_bits -= sizeof(entropy) * 8; > + for (i = 0, arch_bits = sizeof(entropy) * 8; i < ARRAY_SIZE(entropy);) { > + words = arch_get_random_seed_words(entropy, ARRAY_SIZE(entropy) - i); > + if (words) { > + _mix_pool_bytes(entropy, sizeof(*entropy) * words); > + i += words; > + continue; > } > - _mix_pool_bytes(&entropy, sizeof(entropy)); > + words = arch_get_random_words(entropy, ARRAY_SIZE(entropy) - i); > + if (words) { > + _mix_pool_bytes(entropy, sizeof(*entropy) * words); > + i += words; > + continue; > + } > + entropy[0] = random_get_entropy(); > + _mix_pool_bytes(entropy, sizeof(*entropy)); > + arch_bits -= sizeof(*entropy) * 8; > + ++i; > } > _mix_pool_bytes(&now, sizeof(now)); > _mix_pool_bytes(utsname(), sizeof(*(utsname()))); > diff --git a/include/asm-generic/archrandom.h b/include/asm-generic/archrandom.h > index 3a5ee202dd86..ae618916c74c 100644 > --- a/include/asm-generic/archrandom.h > +++ b/include/asm-generic/archrandom.h > @@ -2,24 +2,14 @@ > #ifndef __ASM_GENERIC_ARCHRANDOM_H__ > #define __ASM_GENERIC_ARCHRANDOM_H__ > > -static inline bool __must_check arch_get_random_long(unsigned long *v) > +static inline size_t __must_check arch_get_random_words(unsigned long *v, size_t words) > { > - return false; > + return 0; > } > > -static inline bool __must_check arch_get_random_int(unsigned int *v) > +static inline size_t __must_check arch_get_random_seed_words(unsigned long *v, size_t words) > { > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_long(unsigned long *v) > -{ > - return false; > -} > - > -static inline bool __must_check arch_get_random_seed_int(unsigned int *v) > -{ > - return false; > + return 0; > } > > #endif > diff --git a/include/linux/random.h b/include/linux/random.h > index 865770e29f3e..0a327a289f09 100644 > --- a/include/linux/random.h > +++ b/include/linux/random.h > @@ -112,19 +112,19 @@ declare_get_random_var_wait(long, unsigned long) > * Called from the boot CPU during startup; not valid to call once > * secondary CPUs are up and preemption is possible. > */ > -#ifndef arch_get_random_seed_long_early > -static inline bool __init arch_get_random_seed_long_early(unsigned long *v) > +#ifndef arch_get_random_seed_words_early > +static inline size_t __init arch_get_random_seed_words_early(unsigned long *v, size_t words) > { > WARN_ON(system_state != SYSTEM_BOOTING); > - return arch_get_random_seed_long(v); > + return arch_get_random_seed_words(v, words); > } > #endif > > -#ifndef arch_get_random_long_early > -static inline bool __init arch_get_random_long_early(unsigned long *v) > +#ifndef arch_get_random_words_early > +static inline bool __init arch_get_random_words_early(unsigned long *v, size_t words) > { > WARN_ON(system_state != SYSTEM_BOOTING); > - return arch_get_random_long(v); > + return arch_get_random_words(v, words); > } > #endif > -- Mit freundlichen Grüßen / Kind regards Holger Dengler -- IBM Systems, Linux on IBM Z Development dengler@linux.ibm.com ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2] random: handle archrandom in plural words 2022-07-22 8:08 ` [PATCH v2] random: handle archrandom in plural words Holger Dengler @ 2022-07-22 11:22 ` Jason A. Donenfeld 2022-08-03 12:01 ` Holger Dengler 0 siblings, 1 reply; 27+ messages in thread From: Jason A. Donenfeld @ 2022-07-22 11:22 UTC (permalink / raw) To: Holger Dengler Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Michael Ellerman, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Harald Freudenberger Hi Holger, On Fri, Jul 22, 2022 at 10:08:05AM +0200, Holger Dengler wrote: > Why not changing the API to take bytes instead of words? Sure, at the > moment it looks like all platforms with TRNG support are able to > deliver at least one word, but bytes would be more flexible. The idea is to strike a sweet spot between capabilities. S390x is fine with byte-level granularity up to arbitrary lengths, while x86 is best with word-level granularity of length 1. The happy intersection between the two is just word-level granularity of arbitrary length. Yes we _could_ introduce a lot of code complexity by cascading the x86 case down into smaller and smaller registers, ignoring the fact that it's no longer efficient below 32- or 64-bit registers depending on vendor. But then we're relying on the inliner to remove all of that extra code, since all callers actually only ever want 32 or 64 bytes. Why bloat for nothing? The beauty of this approach is that it translates very naturally over all the various quirks of architectures without having to have a lot of coupling code. The other reason is that it's simply not necessary. The primary use for this in random.c is to fill a 32- or 64-*byte* block with "some stuff", preferring RDSEED, then RDRAND, and finally falling back to RDTSC. These correspond with arch_get_random_seed_longs(), arch_get_random_longs(), and random_get_entropy() (which is usually get_cycles() underneath), respectively. With the cycle counter being (at least) ~word-sized on all platforms, keeping the granularity of the arch_get_random_*_longs() functions the same lets us fill these with a basic cascade that doesn't require a lot of code: unsigned long array[whatever]; for (i = 0; i < ARRAY_SIZE(array);) { longs = arch_get_random_seed_longs(&array[i], ARRAY_SIZE(array) - i); if (longs) { i += longs; continue; } longs = arch_get_random_longs(&array[i], ARRAY_SIZE(array) - i); if (longs) { i += longs; continue; } array[i++] = random_get_entropy(); } By using a word as the underlying unit, the above cascade generates optimal code on basically all archrandom platforms, no matter what their byte-vs-word or one-vs-three-vs-many semantics are. That's a bit long winded, but hopefully that gives a bit of insight on why going from _long -> _longs is so "lazy" looking. Jason ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2] random: handle archrandom in plural words 2022-07-22 11:22 ` Jason A. Donenfeld @ 2022-08-03 12:01 ` Holger Dengler 0 siblings, 0 replies; 27+ messages in thread From: Holger Dengler @ 2022-08-03 12:01 UTC (permalink / raw) To: Jason A. Donenfeld Cc: linux-kernel, linux-arm-kernel, linuxppc-dev, linux-s390, x86, Will Deacon, Michael Ellerman, Alexander Gordeev, Thomas Gleixner, H . Peter Anvin, Catalin Marinas, Borislav Petkov, Heiko Carstens, Johannes Berg, Harald Freudenberger Hi Jason, On 22/07/2022 13:22, Jason A. Donenfeld wrote: > On Fri, Jul 22, 2022 at 10:08:05AM +0200, Holger Dengler wrote: >> Why not changing the API to take bytes instead of words? Sure, at the >> moment it looks like all platforms with TRNG support are able to >> deliver at least one word, but bytes would be more flexible. > > The idea is to strike a sweet spot between capabilities. S390x is fine > with byte-level granularity up to arbitrary lengths, while x86 is best > with word-level granularity of length 1. The happy intersection between > the two is just word-level granularity of arbitrary length. Yes we > _could_ introduce a lot of code complexity by cascading the x86 case > down into smaller and smaller registers, ignoring the fact that it's no > longer efficient below 32- or 64-bit registers depending on vendor. But > then we're relying on the inliner to remove all of that extra code, > since all callers actually only ever want 32 or 64 bytes. Why bloat for > nothing? The beauty of this approach is that it translates very > naturally over all the various quirks of architectures without having to > have a lot of coupling code. You're absolutely right. Your solution addresses all needs of current architectures. My proposal was just meant as preparation for the case, that new (smaller) architectures may come up in the future with a TRNG support, but with other granularity. But anyhow: we can handle it as soon as it happens, fine with me. -- Mit freundlichen Grüßen / Kind regards Holger Dengler -- IBM Systems, Linux on IBM Z Development dengler@linux.ibm.com ^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2022-08-03 12:03 UTC | newest] Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-07-17 11:24 [PATCH] random: handle archrandom in plural words Jason A. Donenfeld 2022-07-17 11:43 ` Jason A. Donenfeld 2022-07-17 20:03 ` [PATCH v2] " Jason A. Donenfeld 2022-07-18 6:31 ` Michael Ellerman 2022-07-18 6:46 ` Gabriel Paubert 2022-07-20 3:03 ` Michael Ellerman 2022-07-19 12:42 ` Mark Rutland 2022-07-19 12:46 ` Jason A. Donenfeld 2022-07-19 13:02 ` [PATCH v3] random: handle archrandom with multiple longs Jason A. Donenfeld 2022-07-19 13:17 ` Mark Rutland 2022-07-19 13:48 ` Catalin Marinas 2022-07-22 12:06 ` Heiko Carstens 2022-07-24 22:47 ` Jason A. Donenfeld 2022-07-25 9:19 ` Borislav Petkov 2022-07-25 9:26 ` Jason A. Donenfeld 2022-07-25 9:36 ` David Laight 2022-07-25 9:37 ` Jason A. Donenfeld 2022-07-25 11:10 ` [PATCH] random: discourage use of archrandom outside of rng Jason A. Donenfeld 2022-07-25 11:25 ` Borislav Petkov 2022-07-25 11:33 ` Heiko Carstens 2022-07-25 13:02 ` Mark Rutland 2022-07-25 11:25 ` [PATCH v3] random: handle archrandom with multiple longs Borislav Petkov 2022-08-01 14:46 ` Harald Freudenberger 2022-08-01 14:50 ` Jason A. Donenfeld 2022-07-22 8:08 ` [PATCH v2] random: handle archrandom in plural words Holger Dengler 2022-07-22 11:22 ` Jason A. Donenfeld 2022-08-03 12:01 ` Holger Dengler
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).