From: Herbert Xu <herbert@gondor.apana.org.au> To: "Jason A. Donenfeld" <Jason@zx2c4.com>, Eric Biggers <ebiggers@kernel.org>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Linux Crypto Mailing List <linux-crypto@vger.kernel.org>, linux-fscrypt@vger.kernel.org, linux-arm-kernel@lists.infradead.org, LKML <linux-kernel@vger.kernel.org>, Paul Crowley <paulcrowley@google.com>, Greg Kaiser <gkaiser@google.com>, Samuel Neves <samuel.c.p.neves@gmail.com>, Tomer Ashur <tomer.ashur@esat.kuleuven.be>, Martin Willi <martin@strongswan.org> Subject: [v2 PATCH 3/4] zinc: Add x86 accelerated ChaCha20 Date: Tue, 20 Nov 2018 14:04:47 +0800 [thread overview] Message-ID: <E1gOz9T-00066y-BK@gondobar> (raw) In-Reply-To: 20181120060217.t4nccaqpwnxkl4tx@gondor.apana.org.au This patch exposes the crypto API x86 chacha20 implementation through zinc. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> --- lib/zinc/Kconfig | 1 lib/zinc/chacha20/chacha20-x86_64-glue.c | 55 +++++++++++++++++++++++++++++++ lib/zinc/chacha20/chacha20.c | 4 ++ 3 files changed, 60 insertions(+) diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig index 1fffd0a1a74c..010547fa6c9d 100644 --- a/lib/zinc/Kconfig +++ b/lib/zinc/Kconfig @@ -1,6 +1,7 @@ config ZINC_CHACHA20 tristate select CRYPTO_CHACHA20 + select CRYPTO_CHACHA20_X86_64 if ZINC_ARCH_X86_64 config ZINC_SELFTEST bool "Zinc cryptography library self-tests" diff --git a/lib/zinc/chacha20/chacha20-x86_64-glue.c b/lib/zinc/chacha20/chacha20-x86_64-glue.c new file mode 100644 index 000000000000..07f72729a64e --- /dev/null +++ b/lib/zinc/chacha20/chacha20-x86_64-glue.c @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved. + */ + +#include <asm/fpu/api.h> +#include <asm/cpufeature.h> +#include <asm/processor.h> +#include <asm/intel-family.h> +#include <crypto/chacha20.h> + +static bool chacha20_use_ssse3 __ro_after_init; +static bool *const chacha20_nobs[] __initconst = { + &chacha20_use_ssse3 }; + +static void __init chacha20_fpu_init(void) +{ + chacha20_use_ssse3 = boot_cpu_has(X86_FEATURE_SSSE3); +} + +static inline bool chacha20_arch(struct chacha20_ctx *ctx, u8 *dst, + const u8 *src, size_t len, + simd_context_t *simd_context) +{ + /* SIMD disables preemption, so relax after processing each page. */ + BUILD_BUG_ON(PAGE_SIZE < CHACHA20_BLOCK_SIZE || + PAGE_SIZE % CHACHA20_BLOCK_SIZE); + + if (!IS_ENABLED(CONFIG_AS_SSSE3) || !chacha20_use_ssse3 || + len <= CHACHA20_BLOCK_SIZE || !simd_use(simd_context)) + return false; + + for (;;) { + const size_t bytes = min_t(size_t, len, PAGE_SIZE); + + crypto_chacha20_dosimd(ctx->state, dst, src, bytes); + + len -= bytes; + if (!len) + break; + dst += bytes; + src += bytes; + simd_relax(simd_context); + } + + return true; +} + +static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], + const u8 nonce[HCHACHA20_NONCE_SIZE], + const u8 key[HCHACHA20_KEY_SIZE], + simd_context_t *simd_context) +{ + return false; +} diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c index 132850d19e39..480d304cd917 100644 --- a/lib/zinc/chacha20/chacha20.c +++ b/lib/zinc/chacha20/chacha20.c @@ -17,6 +17,9 @@ #include <crypto/algapi.h> // For crypto_xor_cpy. #include <crypto/chacha20.h> +#if defined(CONFIG_ZINC_ARCH_X86_64) +#include "chacha20-x86_64-glue.c" +#else static bool *const chacha20_nobs[] __initconst = { }; static void __init chacha20_fpu_init(void) { @@ -34,6 +37,7 @@ static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], { return false; } +#endif void chacha20(struct chacha20_ctx *ctx, u8 *dst, const u8 *src, u32 len, simd_context_t *simd_context)
WARNING: multiple messages have this Message-ID (diff)
From: herbert@gondor.apana.org.au (Herbert Xu) To: linux-arm-kernel@lists.infradead.org Subject: [v2 PATCH 3/4] zinc: Add x86 accelerated ChaCha20 Date: Tue, 20 Nov 2018 14:04:47 +0800 [thread overview] Message-ID: <E1gOz9T-00066y-BK@gondobar> (raw) In-Reply-To: 20181120060217.t4nccaqpwnxkl4tx@gondor.apana.org.au This patch exposes the crypto API x86 chacha20 implementation through zinc. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> --- lib/zinc/Kconfig | 1 lib/zinc/chacha20/chacha20-x86_64-glue.c | 55 +++++++++++++++++++++++++++++++ lib/zinc/chacha20/chacha20.c | 4 ++ 3 files changed, 60 insertions(+) diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig index 1fffd0a1a74c..010547fa6c9d 100644 --- a/lib/zinc/Kconfig +++ b/lib/zinc/Kconfig @@ -1,6 +1,7 @@ config ZINC_CHACHA20 tristate select CRYPTO_CHACHA20 + select CRYPTO_CHACHA20_X86_64 if ZINC_ARCH_X86_64 config ZINC_SELFTEST bool "Zinc cryptography library self-tests" diff --git a/lib/zinc/chacha20/chacha20-x86_64-glue.c b/lib/zinc/chacha20/chacha20-x86_64-glue.c new file mode 100644 index 000000000000..07f72729a64e --- /dev/null +++ b/lib/zinc/chacha20/chacha20-x86_64-glue.c @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved. + */ + +#include <asm/fpu/api.h> +#include <asm/cpufeature.h> +#include <asm/processor.h> +#include <asm/intel-family.h> +#include <crypto/chacha20.h> + +static bool chacha20_use_ssse3 __ro_after_init; +static bool *const chacha20_nobs[] __initconst = { + &chacha20_use_ssse3 }; + +static void __init chacha20_fpu_init(void) +{ + chacha20_use_ssse3 = boot_cpu_has(X86_FEATURE_SSSE3); +} + +static inline bool chacha20_arch(struct chacha20_ctx *ctx, u8 *dst, + const u8 *src, size_t len, + simd_context_t *simd_context) +{ + /* SIMD disables preemption, so relax after processing each page. */ + BUILD_BUG_ON(PAGE_SIZE < CHACHA20_BLOCK_SIZE || + PAGE_SIZE % CHACHA20_BLOCK_SIZE); + + if (!IS_ENABLED(CONFIG_AS_SSSE3) || !chacha20_use_ssse3 || + len <= CHACHA20_BLOCK_SIZE || !simd_use(simd_context)) + return false; + + for (;;) { + const size_t bytes = min_t(size_t, len, PAGE_SIZE); + + crypto_chacha20_dosimd(ctx->state, dst, src, bytes); + + len -= bytes; + if (!len) + break; + dst += bytes; + src += bytes; + simd_relax(simd_context); + } + + return true; +} + +static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], + const u8 nonce[HCHACHA20_NONCE_SIZE], + const u8 key[HCHACHA20_KEY_SIZE], + simd_context_t *simd_context) +{ + return false; +} diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c index 132850d19e39..480d304cd917 100644 --- a/lib/zinc/chacha20/chacha20.c +++ b/lib/zinc/chacha20/chacha20.c @@ -17,6 +17,9 @@ #include <crypto/algapi.h> // For crypto_xor_cpy. #include <crypto/chacha20.h> +#if defined(CONFIG_ZINC_ARCH_X86_64) +#include "chacha20-x86_64-glue.c" +#else static bool *const chacha20_nobs[] __initconst = { }; static void __init chacha20_fpu_init(void) { @@ -34,6 +37,7 @@ static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], { return false; } +#endif void chacha20(struct chacha20_ctx *ctx, u8 *dst, const u8 *src, u32 len, simd_context_t *simd_context)
next prev parent reply other threads:[~2018-11-20 16:32 UTC|newest] Thread overview: 98+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-11-05 23:25 [RFC PATCH v3 00/15] crypto: Adiantum support Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 01/15] crypto: chacha20-generic - add HChaCha20 library function Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 02/15] crypto: chacha20-generic - don't unnecessarily use atomic walk Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 03/15] crypto: chacha20-generic - add XChaCha20 support Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 04/15] crypto: chacha20-generic - refactor to allow varying number of rounds Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 05/15] crypto: chacha - add XChaCha12 support Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 06/15] crypto: arm/chacha20 - limit the preemption-disabled section Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 07/15] crypto: arm/chacha20 - add XChaCha20 support Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-06 12:41 ` Ard Biesheuvel 2018-11-06 12:41 ` Ard Biesheuvel 2018-11-06 12:41 ` Ard Biesheuvel 2018-11-05 23:25 ` [RFC PATCH v3 08/15] crypto: arm/chacha20 - refactor to allow varying number of rounds Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-06 12:46 ` Ard Biesheuvel 2018-11-06 12:46 ` Ard Biesheuvel 2018-11-06 12:46 ` Ard Biesheuvel 2018-11-05 23:25 ` [RFC PATCH v3 09/15] crypto: arm/chacha - add XChaCha12 support Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 10/15] crypto: poly1305 - use structures for key and accumulator Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-06 14:28 ` Ard Biesheuvel 2018-11-06 14:28 ` Ard Biesheuvel 2018-11-06 14:28 ` Ard Biesheuvel 2018-11-12 18:58 ` Eric Biggers 2018-11-12 18:58 ` Eric Biggers 2018-11-12 18:58 ` Eric Biggers 2018-11-16 6:02 ` Herbert Xu 2018-11-16 6:02 ` Herbert Xu 2018-11-16 6:02 ` Herbert Xu 2018-11-17 0:17 ` Eric Biggers 2018-11-17 0:17 ` Eric Biggers 2018-11-17 0:17 ` Eric Biggers 2018-11-17 0:30 ` Ard Biesheuvel 2018-11-17 0:30 ` Ard Biesheuvel 2018-11-17 0:30 ` Ard Biesheuvel 2018-11-18 13:46 ` Jason A. Donenfeld 2018-11-18 13:46 ` Jason A. Donenfeld 2018-11-19 5:24 ` [RFC PATCH] zinc chacha20 generic implementation using crypto API code Herbert Xu 2018-11-19 6:13 ` Jason A. Donenfeld 2018-11-19 6:13 ` Jason A. Donenfeld 2018-11-19 6:22 ` Herbert Xu 2018-11-19 6:22 ` Herbert Xu 2018-11-19 22:54 ` Eric Biggers 2018-11-19 22:54 ` Eric Biggers 2018-11-19 23:15 ` Jason A. Donenfeld 2018-11-19 23:15 ` Jason A. Donenfeld 2018-11-19 23:23 ` Eric Biggers 2018-11-19 23:23 ` Eric Biggers 2018-11-19 23:31 ` Jason A. Donenfeld 2018-11-19 23:31 ` Jason A. Donenfeld 2018-11-20 3:06 ` Herbert Xu 2018-11-20 3:06 ` Herbert Xu 2018-11-20 3:08 ` Jason A. Donenfeld 2018-11-20 3:08 ` Jason A. Donenfeld 2018-11-20 6:02 ` [RFC PATCH v2 0/4] Exporting existing crypto API code through zinc Herbert Xu 2018-11-20 6:02 ` Herbert Xu 2018-11-20 6:04 ` [v2 PATCH 1/4] crypto: chacha20 - Export chacha20 functions without crypto API Herbert Xu 2018-11-20 6:04 ` Herbert Xu 2018-11-20 6:04 ` [v2 PATCH 2/4] zinc: ChaCha20 generic C implementation and selftest Herbert Xu 2018-11-20 6:04 ` Herbert Xu [this message] 2018-11-20 6:04 ` [v2 PATCH 3/4] zinc: Add x86 accelerated ChaCha20 Herbert Xu 2018-11-20 6:04 ` [v2 PATCH 4/4] zinc: ChaCha20 x86_64 implementation Herbert Xu 2018-11-20 10:32 ` [RFC PATCH v2 0/4] Exporting existing crypto API code through zinc Ard Biesheuvel 2018-11-20 10:32 ` Ard Biesheuvel 2018-11-20 10:32 ` Ard Biesheuvel 2018-11-20 14:18 ` Herbert Xu 2018-11-20 14:18 ` Herbert Xu 2018-11-20 14:18 ` Herbert Xu 2018-11-20 16:24 ` Jason A. Donenfeld 2018-11-20 16:24 ` Jason A. Donenfeld 2018-11-20 18:51 ` Theodore Y. Ts'o 2018-11-20 18:51 ` Theodore Y. Ts'o 2018-11-21 7:55 ` Herbert Xu 2018-11-21 7:55 ` Herbert Xu 2018-11-20 16:18 ` Jason A. Donenfeld 2018-11-20 16:18 ` Jason A. Donenfeld 2018-11-21 6:01 ` Herbert Xu 2018-11-21 6:01 ` Herbert Xu 2018-11-05 23:25 ` [RFC PATCH v3 11/15] crypto: poly1305 - add Poly1305 core API Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 12/15] crypto: nhpoly1305 - add NHPoly1305 support Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 13/15] crypto: arm/nhpoly1305 - add NEON-accelerated NHPoly1305 Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 14/15] crypto: adiantum - add Adiantum support Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-05 23:25 ` [RFC PATCH v3 15/15] fscrypt: " Eric Biggers 2018-11-05 23:25 ` Eric Biggers 2018-11-08 6:47 ` [RFC PATCH v3 00/15] crypto: " Martin Willi 2018-11-08 6:47 ` Martin Willi
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=E1gOz9T-00066y-BK@gondobar \ --to=herbert@gondor.apana.org.au \ --cc=Jason@zx2c4.com \ --cc=ard.biesheuvel@linaro.org \ --cc=ebiggers@kernel.org \ --cc=gkaiser@google.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-crypto@vger.kernel.org \ --cc=linux-fscrypt@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=martin@strongswan.org \ --cc=paulcrowley@google.com \ --cc=samuel.c.p.neves@gmail.com \ --cc=tomer.ashur@esat.kuleuven.be \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.