From: Robert Elliott <elliott@hpe.com>
To: herbert@gondor.apana.org.au, davem@davemloft.net,
tim.c.chen@linux.intel.com, ap420073@gmail.com, ardb@kernel.org,
Jason@zx2c4.com, David.Laight@ACULAB.COM, ebiggers@kernel.org,
linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: Robert Elliott <elliott@hpe.com>
Subject: [PATCH v3 17/17] crypto: x86/nhpoly1305, poly1305 - load based on CPU features
Date: Wed, 2 Nov 2022 23:27:40 -0500 [thread overview]
Message-ID: <20221103042740.6556-18-elliott@hpe.com> (raw)
In-Reply-To: <20221103042740.6556-1-elliott@hpe.com>
Like commit aa031b8f702e ("crypto: x86/sha512 - load based on CPU
features"), add module aliases for x86-optimized crypto modules:
nhpoly1305, poly1305
based on CPU feature bits so udev gets a chance to load them later
in the boot process when the filesystems are all running.
Signed-off-by: Robert Elliott <elliott@hpe.com>
---
arch/x86/crypto/nhpoly1305-avx2-glue.c | 10 ++++++++++
arch/x86/crypto/nhpoly1305-sse2-glue.c | 10 ++++++++++
arch/x86/crypto/poly1305_glue.c | 12 ++++++++++++
3 files changed, 32 insertions(+)
diff --git a/arch/x86/crypto/nhpoly1305-avx2-glue.c b/arch/x86/crypto/nhpoly1305-avx2-glue.c
index f7dc9c563bb5..15f98b53bfda 100644
--- a/arch/x86/crypto/nhpoly1305-avx2-glue.c
+++ b/arch/x86/crypto/nhpoly1305-avx2-glue.c
@@ -11,6 +11,7 @@
#include <crypto/nhpoly1305.h>
#include <linux/module.h>
#include <linux/sizes.h>
+#include <asm/cpu_device_id.h>
#include <asm/simd.h>
/* avoid kernel_fpu_begin/end scheduler/rcu stalls */
@@ -60,8 +61,17 @@ static struct shash_alg nhpoly1305_alg = {
.descsize = sizeof(struct nhpoly1305_state),
};
+static const struct x86_cpu_id module_cpu_ids[] = {
+ X86_MATCH_FEATURE(X86_FEATURE_AVX2, NULL),
+ {}
+};
+MODULE_DEVICE_TABLE(x86cpu, module_cpu_ids);
+
static int __init nhpoly1305_mod_init(void)
{
+ if (!x86_match_cpu(module_cpu_ids))
+ return -ENODEV;
+
if (!boot_cpu_has(X86_FEATURE_AVX2) ||
!boot_cpu_has(X86_FEATURE_OSXSAVE))
return -ENODEV;
diff --git a/arch/x86/crypto/nhpoly1305-sse2-glue.c b/arch/x86/crypto/nhpoly1305-sse2-glue.c
index daffcc7019ad..533db3e0e06f 100644
--- a/arch/x86/crypto/nhpoly1305-sse2-glue.c
+++ b/arch/x86/crypto/nhpoly1305-sse2-glue.c
@@ -11,6 +11,7 @@
#include <crypto/nhpoly1305.h>
#include <linux/module.h>
#include <linux/sizes.h>
+#include <asm/cpu_device_id.h>
#include <asm/simd.h>
/* avoid kernel_fpu_begin/end scheduler/rcu stalls */
@@ -60,8 +61,17 @@ static struct shash_alg nhpoly1305_alg = {
.descsize = sizeof(struct nhpoly1305_state),
};
+static const struct x86_cpu_id module_cpu_ids[] = {
+ X86_MATCH_FEATURE(X86_FEATURE_XMM2, NULL),
+ {}
+};
+MODULE_DEVICE_TABLE(x86cpu, module_cpu_ids);
+
static int __init nhpoly1305_mod_init(void)
{
+ if (!x86_match_cpu(module_cpu_ids))
+ return -ENODEV;
+
if (!boot_cpu_has(X86_FEATURE_XMM2))
return -ENODEV;
diff --git a/arch/x86/crypto/poly1305_glue.c b/arch/x86/crypto/poly1305_glue.c
index 16831c036d71..2ff4358e4b3f 100644
--- a/arch/x86/crypto/poly1305_glue.c
+++ b/arch/x86/crypto/poly1305_glue.c
@@ -12,6 +12,7 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/sizes.h>
+#include <asm/cpu_device_id.h>
#include <asm/intel-family.h>
#include <asm/simd.h>
@@ -268,8 +269,19 @@ static struct shash_alg alg = {
},
};
+static const struct x86_cpu_id module_cpu_ids[] = {
+ X86_MATCH_FEATURE(X86_FEATURE_AVX2, NULL),
+ X86_MATCH_FEATURE(X86_FEATURE_AVX, NULL),
+ X86_MATCH_FEATURE(X86_FEATURE_AVX512F, NULL),
+ {}
+};
+MODULE_DEVICE_TABLE(x86cpu, module_cpu_ids);
+
static int __init poly1305_simd_mod_init(void)
{
+ if (!x86_match_cpu(module_cpu_ids))
+ return -ENODEV;
+
if (boot_cpu_has(X86_FEATURE_AVX) &&
cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL))
static_branch_enable(&poly1305_use_avx);
--
2.37.3
next prev parent reply other threads:[~2022-11-03 4:30 UTC|newest]
Thread overview: 126+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-06 22:31 [RFC PATCH 0/7] crypto: x86 - fix RCU stalls Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 1/7] rcu: correct CONFIG_EXT_RCU_CPU_STALL_TIMEOUT descriptions Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 2/7] crypto: x86/sha - limit FPU preemption Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 3/7] crypto: x86/crc " Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 4/7] crypto: x86/sm3 " Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 5/7] crypto: x86/ghash - restructure FPU context saving Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 6/7] crypto: x86/ghash - limit FPU preemption Robert Elliott
2022-10-06 22:31 ` [RFC PATCH 7/7] crypto: x86 - use common macro for FPU limit Robert Elliott
2022-10-12 21:59 ` [PATCH v2 00/19] crypto: x86 - fix RCU stalls Robert Elliott
2022-10-12 21:59 ` [PATCH v2 01/19] crypto: tcrypt - test crc32 Robert Elliott
2022-10-12 21:59 ` [PATCH v2 02/19] crypto: tcrypt - test nhpoly1305 Robert Elliott
2022-10-12 21:59 ` [PATCH v2 03/19] crypto: tcrypt - reschedule during cycles speed tests Robert Elliott
2022-10-12 21:59 ` [PATCH v2 04/19] crypto: x86/sha - limit FPU preemption Robert Elliott
2022-10-13 0:41 ` Jason A. Donenfeld
2022-10-13 21:50 ` Elliott, Robert (Servers)
2022-10-14 11:01 ` David Laight
2022-10-13 5:57 ` Eric Biggers
2022-10-13 6:04 ` Herbert Xu
2022-10-13 6:08 ` Eric Biggers
2022-10-13 7:50 ` Herbert Xu
2022-10-13 22:41 ` :Re: " Elliott, Robert (Servers)
2022-10-12 21:59 ` [PATCH v2 05/19] crypto: x86/crc " Robert Elliott
2022-10-13 2:00 ` Herbert Xu
2022-10-13 22:34 ` Elliott, Robert (Servers)
2022-10-14 4:02 ` David Laight
2022-10-24 2:03 ` kernel test robot
2022-10-12 21:59 ` [PATCH v2 06/19] crypto: x86/sm3 " Robert Elliott
2022-10-12 21:59 ` [PATCH v2 07/19] crypto: x86/ghash - restructure FPU context saving Robert Elliott
2022-10-12 21:59 ` [PATCH v2 08/19] crypto: x86/ghash - limit FPU preemption Robert Elliott
2022-10-13 6:03 ` Eric Biggers
2022-10-13 22:52 ` Elliott, Robert (Servers)
2022-10-12 21:59 ` [PATCH v2 09/19] crypto: x86 - use common macro for FPU limit Robert Elliott
2022-10-13 0:35 ` Jason A. Donenfeld
2022-10-13 21:48 ` Elliott, Robert (Servers)
2022-10-14 1:26 ` Jason A. Donenfeld
2022-10-18 0:06 ` Elliott, Robert (Servers)
2022-10-12 21:59 ` [PATCH v2 10/19] crypto: x86/sha1, sha256 - load based on CPU features Robert Elliott
2022-10-12 21:59 ` [PATCH v2 11/19] crypto: x86/crc " Robert Elliott
2022-10-12 21:59 ` [PATCH v2 12/19] crypto: x86/sm3 " Robert Elliott
2022-10-12 21:59 ` [PATCH v2 13/19] crypto: x86/ghash " Robert Elliott
2022-10-12 21:59 ` [PATCH v2 14/19] crypto: x86 " Robert Elliott
2022-10-14 14:26 ` Elliott, Robert (Servers)
2022-10-12 21:59 ` [PATCH v2 15/19] crypto: x86 - add pr_fmt to all modules Robert Elliott
2022-10-12 21:59 ` [PATCH v2 16/19] crypto: x86 - print CPU optimized loaded messages Robert Elliott
2022-10-13 0:40 ` Jason A. Donenfeld
2022-10-13 13:47 ` kernel test robot
2022-10-13 13:48 ` kernel test robot
2022-10-12 21:59 ` [PATCH v2 17/19] crypto: x86 - standardize suboptimal prints Robert Elliott
2022-10-13 0:38 ` Jason A. Donenfeld
2022-10-12 21:59 ` [PATCH v2 18/19] crypto: x86 - standardize not loaded prints Robert Elliott
2022-10-13 0:42 ` Jason A. Donenfeld
2022-10-13 22:20 ` Elliott, Robert (Servers)
2022-11-10 22:06 ` Elliott, Robert (Servers)
2022-10-12 21:59 ` [PATCH v2 19/19] crypto: x86/sha - register only the best function Robert Elliott
2022-10-13 6:07 ` Eric Biggers
2022-10-13 7:52 ` Herbert Xu
2022-10-13 22:59 ` Elliott, Robert (Servers)
2022-10-14 8:22 ` Herbert Xu
2022-11-01 21:34 ` [PATCH v2 00/19] crypto: x86 - fix RCU stalls Elliott, Robert (Servers)
2022-11-03 4:27 ` [PATCH v3 00/17] crypt: " Robert Elliott
2022-11-03 4:27 ` [PATCH v3 01/17] crypto: tcrypt - test crc32 Robert Elliott
2022-11-03 4:27 ` [PATCH v3 02/17] crypto: tcrypt - test nhpoly1305 Robert Elliott
2022-11-03 4:27 ` [PATCH v3 03/17] crypto: tcrypt - reschedule during cycles speed tests Robert Elliott
2022-11-03 4:27 ` [PATCH v3 04/17] crypto: x86/sha - limit FPU preemption Robert Elliott
2022-11-03 4:27 ` [PATCH v3 05/17] crypto: x86/crc " Robert Elliott
2022-11-03 4:27 ` [PATCH v3 06/17] crypto: x86/sm3 " Robert Elliott
2022-11-03 4:27 ` [PATCH v3 07/17] crypto: x86/ghash - use u8 rather than char Robert Elliott
2022-11-03 4:27 ` [PATCH v3 08/17] crypto: x86/ghash - restructure FPU context saving Robert Elliott
2022-11-03 4:27 ` [PATCH v3 09/17] crypto: x86/ghash - limit FPU preemption Robert Elliott
2022-11-03 4:27 ` [PATCH v3 10/17] crypto: x86/*poly* " Robert Elliott
2022-11-03 4:27 ` [PATCH v3 11/17] crypto: x86/sha - register all variations Robert Elliott
2022-11-03 9:26 ` kernel test robot
2022-11-03 4:27 ` [PATCH v3 12/17] crypto: x86/sha - minimize time in FPU context Robert Elliott
2022-11-03 4:27 ` [PATCH v3 13/17] crypto: x86/sha1, sha256 - load based on CPU features Robert Elliott
2022-11-03 4:27 ` [PATCH v3 14/17] crypto: x86/crc " Robert Elliott
2022-11-03 4:27 ` [PATCH v3 15/17] crypto: x86/sm3 " Robert Elliott
2022-11-03 4:27 ` [PATCH v3 16/17] crypto: x86/ghash,polyval " Robert Elliott
2022-11-03 4:27 ` Robert Elliott [this message]
2022-11-16 4:13 ` [PATCH v4 00/24] crypto: fix RCU stalls Robert Elliott
2022-11-16 4:13 ` [PATCH v4 01/24] crypto: tcrypt - test crc32 Robert Elliott
2022-11-16 4:13 ` [PATCH v4 02/24] crypto: tcrypt - test nhpoly1305 Robert Elliott
2022-11-16 4:13 ` [PATCH v4 03/24] crypto: tcrypt - reschedule during cycles speed tests Robert Elliott
2022-11-16 4:13 ` [PATCH v4 04/24] crypto: x86/sha - limit FPU preemption Robert Elliott
2022-11-16 4:13 ` [PATCH v4 05/24] crypto: x86/crc " Robert Elliott
2022-11-16 4:13 ` [PATCH v4 06/24] crypto: x86/sm3 " Robert Elliott
2022-11-16 4:13 ` [PATCH v4 07/24] crypto: x86/ghash - use u8 rather than char Robert Elliott
2022-11-16 4:13 ` [PATCH v4 08/24] crypto: x86/ghash - restructure FPU context saving Robert Elliott
2022-11-16 4:13 ` [PATCH v4 09/24] crypto: x86/ghash - limit FPU preemption Robert Elliott
2022-11-16 4:13 ` [PATCH v4 10/24] crypto: x86/poly " Robert Elliott
2022-11-16 11:13 ` Jason A. Donenfeld
2022-11-22 5:06 ` Elliott, Robert (Servers)
2022-11-22 9:07 ` David Laight
2022-11-25 8:40 ` Herbert Xu
2022-11-25 8:59 ` Ard Biesheuvel
2022-11-25 9:03 ` Herbert Xu
2022-11-28 16:57 ` Elliott, Robert (Servers)
2022-11-28 18:48 ` Elliott, Robert (Servers)
2022-12-02 6:21 ` Elliott, Robert (Servers)
2022-12-02 9:25 ` Herbert Xu
2022-12-02 16:15 ` Elliott, Robert (Servers)
2022-12-06 4:27 ` Herbert Xu
2022-12-06 14:03 ` Peter Lafreniere
2022-12-06 14:44 ` David Laight
2022-12-06 23:06 ` Peter Lafreniere
2022-12-10 0:34 ` Elliott, Robert (Servers)
2022-12-16 22:12 ` Elliott, Robert (Servers)
2022-11-16 4:13 ` [PATCH v4 11/24] crypto: x86/aegis " Robert Elliott
2022-11-16 4:13 ` [PATCH v4 12/24] crypto: x86/sha - register all variations Robert Elliott
2022-11-16 4:13 ` [PATCH v4 13/24] crypto: x86/sha - minimize time in FPU context Robert Elliott
2022-11-16 4:13 ` [PATCH v4 14/24] crypto: x86/sha - load based on CPU features Robert Elliott
2022-11-16 4:13 ` [PATCH v4 15/24] crypto: x86/crc " Robert Elliott
2022-11-16 4:13 ` [PATCH v4 16/24] crypto: x86/sm3 " Robert Elliott
2022-11-16 4:13 ` [PATCH v4 17/24] crypto: x86/poly " Robert Elliott
2022-11-16 11:19 ` Jason A. Donenfeld
2022-11-16 4:13 ` [PATCH v4 18/24] crypto: x86/ghash " Robert Elliott
2022-11-16 4:13 ` [PATCH v4 19/24] crypto: x86/aesni - avoid type conversions Robert Elliott
2022-11-16 4:13 ` [PATCH v4 20/24] crypto: x86/ciphers - load based on CPU features Robert Elliott
2022-11-16 11:30 ` Jason A. Donenfeld
2022-11-16 4:13 ` [PATCH v4 21/24] crypto: x86 - report used CPU features via module parameters Robert Elliott
2022-11-16 11:26 ` Jason A. Donenfeld
2022-11-16 4:13 ` [PATCH v4 22/24] crypto: x86 - report missing " Robert Elliott
2022-11-16 4:13 ` [PATCH v4 23/24] crypto: x86 - report suboptimal CPUs " Robert Elliott
2022-11-16 4:13 ` [PATCH v4 24/24] crypto: x86 - standarize module descriptions Robert Elliott
2022-11-17 3:58 ` [PATCH v4 00/24] crypto: fix RCU stalls Herbert Xu
2022-11-17 15:13 ` Elliott, Robert (Servers)
2022-11-17 15:15 ` Jason A. Donenfeld
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221103042740.6556-18-elliott@hpe.com \
--to=elliott@hpe.com \
--cc=David.Laight@ACULAB.COM \
--cc=Jason@zx2c4.com \
--cc=ap420073@gmail.com \
--cc=ardb@kernel.org \
--cc=davem@davemloft.net \
--cc=ebiggers@kernel.org \
--cc=herbert@gondor.apana.org.au \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=tim.c.chen@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).