From: Herbert Xu <herbert@gondor.apana.org.au> To: Linus Torvalds <torvalds@linux-foundation.org>, "David S. Miller" <davem@davemloft.net>, "Jason A. Donenfeld" <Jason@zx2c4.com>, Eric Biggers <ebiggers@kernel.org>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Linux Crypto Mailing List <linux-crypto@vger.kernel.org>, linux-fscrypt@vger.kernel.org, linux-arm-kernel@lists.infradead.org, LKML <linux-kernel@vger.kernel.org>, Paul Crowley <paulcrowley@google.com>, Greg Kaiser <gkaiser@google.com>, Samuel Neves <samuel.c.p.neves@gmail.com>, Tomer Ashur <tomer.ashur@esat.kuleuven.be>, Martin Willi <martin@strongswan.org> Subject: [PATCH 5/17] zinc: Add x86 accelerated ChaCha20 Date: Fri, 22 Mar 2019 14:29:42 +0800 [thread overview] Message-ID: <E1h7DgU-0001Hl-4u@gondobar> (raw) In-Reply-To: 20190322062740.nrwfx2rvmt7lzotj@gondor.apana.org.au This patch exposes the crypto API x86 chacha20 implementation through zinc. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> --- lib/zinc/Kconfig | 1 lib/zinc/chacha20/chacha20-x86_64-glue.c | 55 +++++++++++++++++++++++++++++++ lib/zinc/chacha20/chacha20.c | 4 ++ 3 files changed, 60 insertions(+) diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig index 1fffd0a1a74c..010547fa6c9d 100644 --- a/lib/zinc/Kconfig +++ b/lib/zinc/Kconfig @@ -1,6 +1,7 @@ config ZINC_CHACHA20 tristate select CRYPTO_CHACHA20 + select CRYPTO_CHACHA20_X86_64 if ZINC_ARCH_X86_64 config ZINC_SELFTEST bool "Zinc cryptography library self-tests" diff --git a/lib/zinc/chacha20/chacha20-x86_64-glue.c b/lib/zinc/chacha20/chacha20-x86_64-glue.c new file mode 100644 index 000000000000..e0b387b09c04 --- /dev/null +++ b/lib/zinc/chacha20/chacha20-x86_64-glue.c @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved. + */ + +#include <asm/fpu/api.h> +#include <asm/cpufeature.h> +#include <asm/processor.h> +#include <asm/intel-family.h> +#include <crypto/chacha.h> + +static bool chacha20_use_ssse3 __ro_after_init; +static bool *const chacha20_nobs[] __initconst = { + &chacha20_use_ssse3 }; + +static void __init chacha20_fpu_init(void) +{ + chacha20_use_ssse3 = boot_cpu_has(X86_FEATURE_SSSE3); +} + +static inline bool chacha20_arch(struct chacha20_ctx *ctx, u8 *dst, + const u8 *src, size_t len, + simd_context_t *simd_context) +{ + /* SIMD disables preemption, so relax after processing each page. */ + BUILD_BUG_ON(PAGE_SIZE < CHACHA20_BLOCK_SIZE || + PAGE_SIZE % CHACHA20_BLOCK_SIZE); + + if (!IS_ENABLED(CONFIG_AS_SSSE3) || !chacha20_use_ssse3 || + len <= CHACHA20_BLOCK_SIZE || !simd_use(simd_context)) + return false; + + for (;;) { + const size_t bytes = min_t(size_t, len, PAGE_SIZE); + + crypto_chacha_dosimd(ctx->state, dst, src, bytes, 20); + + len -= bytes; + if (!len) + break; + dst += bytes; + src += bytes; + simd_relax(simd_context); + } + + return true; +} + +static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], + const u8 nonce[HCHACHA20_NONCE_SIZE], + const u8 key[HCHACHA20_KEY_SIZE], + simd_context_t *simd_context) +{ + return false; +} diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c index cfbdaf4384dc..c84fe504623f 100644 --- a/lib/zinc/chacha20/chacha20.c +++ b/lib/zinc/chacha20/chacha20.c @@ -17,6 +17,9 @@ #include <crypto/algapi.h> // For crypto_xor_cpy. #include <crypto/chacha.h> +#if defined(CONFIG_ZINC_ARCH_X86_64) +#include "chacha20-x86_64-glue.c" +#else static bool *const chacha20_nobs[] __initconst = { }; static void __init chacha20_fpu_init(void) { @@ -34,6 +37,7 @@ static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], { return false; } +#endif void chacha20(struct chacha20_ctx *ctx, u8 *dst, const u8 *src, u32 len, simd_context_t *simd_context)
WARNING: multiple messages have this Message-ID (diff)
From: Herbert Xu <herbert@gondor.apana.org.au> To: Linus Torvalds <torvalds@linux-foundation.org>, "David S. Miller" <davem@davemloft.net>, "Jason A. Donenfeld" <Jason@zx2c4.com>, Eric Biggers <ebiggers@kernel.org>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Linux Crypto Mailing List <linux-crypto@vger.kernel.org>, linux-fscrypt@vger.kernel.org, linux-arm-kernel@lists.infradead.org, LKML <linux-kernel@vger.kernel.org>, Paul Crowley <paulcrowley@google.com>, Greg Kaiser <gkaiser@google.com>, Samuel Neves <samuel.c.p.neves@gmail.com>, Tomer Ashur <tomer.ashur@esat.kuleuven.be>, Martin Willi <martin@strongswan.org> Subject: [PATCH 5/17] zinc: Add x86 accelerated ChaCha20 Date: Fri, 22 Mar 2019 14:29:42 +0800 [thread overview] Message-ID: <E1h7DgU-0001Hl-4u@gondobar> (raw) In-Reply-To: 20190322062740.nrwfx2rvmt7lzotj@gondor.apana.org.au This patch exposes the crypto API x86 chacha20 implementation through zinc. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> --- lib/zinc/Kconfig | 1 lib/zinc/chacha20/chacha20-x86_64-glue.c | 55 +++++++++++++++++++++++++++++++ lib/zinc/chacha20/chacha20.c | 4 ++ 3 files changed, 60 insertions(+) diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig index 1fffd0a1a74c..010547fa6c9d 100644 --- a/lib/zinc/Kconfig +++ b/lib/zinc/Kconfig @@ -1,6 +1,7 @@ config ZINC_CHACHA20 tristate select CRYPTO_CHACHA20 + select CRYPTO_CHACHA20_X86_64 if ZINC_ARCH_X86_64 config ZINC_SELFTEST bool "Zinc cryptography library self-tests" diff --git a/lib/zinc/chacha20/chacha20-x86_64-glue.c b/lib/zinc/chacha20/chacha20-x86_64-glue.c new file mode 100644 index 000000000000..e0b387b09c04 --- /dev/null +++ b/lib/zinc/chacha20/chacha20-x86_64-glue.c @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved. + */ + +#include <asm/fpu/api.h> +#include <asm/cpufeature.h> +#include <asm/processor.h> +#include <asm/intel-family.h> +#include <crypto/chacha.h> + +static bool chacha20_use_ssse3 __ro_after_init; +static bool *const chacha20_nobs[] __initconst = { + &chacha20_use_ssse3 }; + +static void __init chacha20_fpu_init(void) +{ + chacha20_use_ssse3 = boot_cpu_has(X86_FEATURE_SSSE3); +} + +static inline bool chacha20_arch(struct chacha20_ctx *ctx, u8 *dst, + const u8 *src, size_t len, + simd_context_t *simd_context) +{ + /* SIMD disables preemption, so relax after processing each page. */ + BUILD_BUG_ON(PAGE_SIZE < CHACHA20_BLOCK_SIZE || + PAGE_SIZE % CHACHA20_BLOCK_SIZE); + + if (!IS_ENABLED(CONFIG_AS_SSSE3) || !chacha20_use_ssse3 || + len <= CHACHA20_BLOCK_SIZE || !simd_use(simd_context)) + return false; + + for (;;) { + const size_t bytes = min_t(size_t, len, PAGE_SIZE); + + crypto_chacha_dosimd(ctx->state, dst, src, bytes, 20); + + len -= bytes; + if (!len) + break; + dst += bytes; + src += bytes; + simd_relax(simd_context); + } + + return true; +} + +static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], + const u8 nonce[HCHACHA20_NONCE_SIZE], + const u8 key[HCHACHA20_KEY_SIZE], + simd_context_t *simd_context) +{ + return false; +} diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c index cfbdaf4384dc..c84fe504623f 100644 --- a/lib/zinc/chacha20/chacha20.c +++ b/lib/zinc/chacha20/chacha20.c @@ -17,6 +17,9 @@ #include <crypto/algapi.h> // For crypto_xor_cpy. #include <crypto/chacha.h> +#if defined(CONFIG_ZINC_ARCH_X86_64) +#include "chacha20-x86_64-glue.c" +#else static bool *const chacha20_nobs[] __initconst = { }; static void __init chacha20_fpu_init(void) { @@ -34,6 +37,7 @@ static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], { return false; } +#endif void chacha20(struct chacha20_ctx *ctx, u8 *dst, const u8 *src, u32 len, simd_context_t *simd_context) _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-03-22 6:31 UTC|newest] Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-03-22 6:27 [PATCH 0/17] Add zinc using existing algorithm implementations Herbert Xu 2019-03-22 6:27 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 1/17] asm: simd context helper API Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 2/17] crypto: chacha20 - Export chacha20 functions without crypto API Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 3/17] zinc: introduce minimal cryptography library Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 4/17] zinc: Add generic C implementation of chacha20 and self-test Herbert Xu 2019-03-22 6:29 ` Herbert Xu [this message] 2019-03-22 6:29 ` [PATCH 5/17] zinc: Add x86 accelerated ChaCha20 Herbert Xu 2019-03-22 6:29 ` [PATCH 6/17] zinc: Add arm accelerated chacha20 Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 7/17] crypto: poly1305 - Export core functions without crypto API Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 8/17] zinc: Add generic C implementation of poly1305 and self-test Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 9/17] zinc: Add x86 accelerated poly1305 Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 10/17] zinc: ChaCha20Poly1305 construction and selftest Herbert Xu 2019-03-22 6:29 ` [PATCH 11/17] zinc: BLAKE2s generic C implementation " Herbert Xu 2019-03-22 6:29 ` [PATCH 12/17] zinc: BLAKE2s x86_64 implementation Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 13/17] zinc: Curve25519 generic C implementations and selftest Herbert Xu 2019-03-22 6:29 ` [PATCH 14/17] zinc: Curve25519 x86_64 implementation Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 15/17] zinc: import Bernstein and Schwabe's Curve25519 ARM implementation Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 16/17] zinc: " Herbert Xu 2019-03-22 6:29 ` [PATCH 17/17] security/keys: rewrite big_key crypto to use Zinc Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:41 ` [PATCH 0/17] Add zinc using existing algorithm implementations Jason A. Donenfeld 2019-03-22 6:41 ` Jason A. Donenfeld 2019-03-22 7:56 ` Ard Biesheuvel 2019-03-22 7:56 ` Ard Biesheuvel 2019-03-22 8:10 ` Jason A. Donenfeld 2019-03-22 8:10 ` Jason A. Donenfeld 2019-03-22 17:48 ` Linus Torvalds 2019-03-22 17:48 ` Linus Torvalds 2019-03-25 9:10 ` Pascal Van Leeuwen 2019-03-25 9:10 ` Pascal Van Leeuwen 2019-03-26 9:46 ` Riku Voipio 2019-03-26 9:46 ` Riku Voipio 2019-04-09 16:14 ` Pascal Van Leeuwen 2019-04-09 16:14 ` Pascal Van Leeuwen 2019-04-09 16:14 ` Pascal Van Leeuwen 2019-03-25 10:43 ` Ard Biesheuvel 2019-03-25 10:43 ` Ard Biesheuvel 2019-03-25 10:45 ` Jason A. Donenfeld 2019-03-25 10:45 ` Jason A. Donenfeld
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=E1h7DgU-0001Hl-4u@gondobar \ --to=herbert@gondor.apana.org.au \ --cc=Jason@zx2c4.com \ --cc=ard.biesheuvel@linaro.org \ --cc=davem@davemloft.net \ --cc=ebiggers@kernel.org \ --cc=gkaiser@google.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-crypto@vger.kernel.org \ --cc=linux-fscrypt@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=martin@strongswan.org \ --cc=paulcrowley@google.com \ --cc=samuel.c.p.neves@gmail.com \ --cc=tomer.ashur@esat.kuleuven.be \ --cc=torvalds@linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.