From: Herbert Xu <herbert@gondor.apana.org.au> To: Linus Torvalds <torvalds@linux-foundation.org>, "David S. Miller" <davem@davemloft.net>, "Jason A. Donenfeld" <Jason@zx2c4.com>, Eric Biggers <ebiggers@kernel.org>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Linux Crypto Mailing List <linux-crypto@vger.kernel.org>, linux-fscrypt@vger.kernel.org, linux-arm-kernel@lists.infradead.org, LKML <linux-kernel@vger.kernel.org>, Paul Crowley <paulcrowley@google.com>, Greg Kaiser <gkaiser@google.com>, Samuel Neves <samuel.c.p.neves@gmail.com>, Tomer Ashur <tomer.ashur@esat.kuleuven.be>, Martin Willi <martin@strongswan.org> Subject: [PATCH 9/17] zinc: Add x86 accelerated poly1305 Date: Fri, 22 Mar 2019 14:29:47 +0800 [thread overview] Message-ID: <E1h7DgZ-0001IW-E8@gondobar> (raw) In-Reply-To: 20190322062740.nrwfx2rvmt7lzotj@gondor.apana.org.au This patch exposes the crypto API x86 poly1305 implementation through zinc. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> --- lib/zinc/Kconfig | 1 lib/zinc/poly1305/poly1305-x86_64-glue.c | 68 +++++++++++++++++++++++++++++++ lib/zinc/poly1305/poly1305.c | 4 + 3 files changed, 73 insertions(+) diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig index 70d45b3288d2..34fa49beb37f 100644 --- a/lib/zinc/Kconfig +++ b/lib/zinc/Kconfig @@ -8,6 +8,7 @@ config ZINC_CHACHA20 config ZINC_POLY1305 tristate select CRYPTO_POLY1305 + select CRYPTO_POLY1305_X86_64 if ZINC_ARCH_X86_64 config ZINC_SELFTEST bool "Zinc cryptography library self-tests" diff --git a/lib/zinc/poly1305/poly1305-x86_64-glue.c b/lib/zinc/poly1305/poly1305-x86_64-glue.c new file mode 100644 index 000000000000..9e6dbd40911d --- /dev/null +++ b/lib/zinc/poly1305/poly1305-x86_64-glue.c @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved. + */ + +#include <asm/cpufeature.h> +#include <asm/processor.h> +#include <asm/intel-family.h> + +static bool poly1305_use_sse2 __ro_after_init; +static bool *const poly1305_nobs[] __initconst = { + &poly1305_use_sse2 }; + +static void __init poly1305_fpu_init(void) +{ + poly1305_use_sse2 = boot_cpu_has(X86_FEATURE_XMM2); +} + +struct poly1305_arch_internal { + struct poly1305_internal base; + struct poly1305_simd_xtra x; +}; + +static inline bool poly1305_init_arch(void *ctx, + const u8 key[POLY1305_KEY_SIZE]) +{ + struct poly1305_arch_internal *state = ctx; + + poly1305_simd_init(&state->x); + poly1305_core_setkey(&state->base.r, key); + return true; +} + +static inline bool poly1305_blocks_arch(void *ctx, const u8 *inp, + size_t len, const u32 padbit, + simd_context_t *simd_context) +{ + struct poly1305_arch_internal *state = ctx; + + /* SIMD disables preemption, so relax after processing each page. */ + BUILD_BUG_ON(PAGE_SIZE < POLY1305_BLOCK_SIZE || + PAGE_SIZE % POLY1305_BLOCK_SIZE); + + if (!poly1305_use_sse2 || !simd_use(simd_context) || padbit) + return false; + + for (;;) { + const size_t bytes = min_t(size_t, len, PAGE_SIZE); + + poly1305_simd_blocks(&state->base.h, &state->base.r, + &state->x, inp, bytes); + + len -= bytes; + if (!len) + break; + inp += bytes; + simd_relax(simd_context); + } + + return true; +} + +static inline bool poly1305_emit_arch(void *ctx, u8 mac[POLY1305_MAC_SIZE], + const u32 nonce[4], + simd_context_t *simd_context) +{ + return false; +} diff --git a/lib/zinc/poly1305/poly1305.c b/lib/zinc/poly1305/poly1305.c index 1546f5ac6d1c..c6a88136abb7 100644 --- a/lib/zinc/poly1305/poly1305.c +++ b/lib/zinc/poly1305/poly1305.c @@ -22,6 +22,9 @@ struct poly1305_internal { struct poly1305_state h; }; +#if defined(CONFIG_ZINC_ARCH_X86_64) +#include "poly1305-x86_64-glue.c" +#else static inline bool poly1305_init_arch(void *ctx, const u8 key[POLY1305_KEY_SIZE]) { @@ -43,6 +46,7 @@ static bool *const poly1305_nobs[] __initconst = { }; static void __init poly1305_fpu_init(void) { } +#endif static void poly1305_init_generic(void *ctx, const u8 key[16]) {
WARNING: multiple messages have this Message-ID (diff)
From: Herbert Xu <herbert@gondor.apana.org.au> To: Linus Torvalds <torvalds@linux-foundation.org>, "David S. Miller" <davem@davemloft.net>, "Jason A. Donenfeld" <Jason@zx2c4.com>, Eric Biggers <ebiggers@kernel.org>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Linux Crypto Mailing List <linux-crypto@vger.kernel.org>, linux-fscrypt@vger.kernel.org, linux-arm-kernel@lists.infradead.org, LKML <linux-kernel@vger.kernel.org>, Paul Crowley <paulcrowley@google.com>, Greg Kaiser <gkaiser@google.com>, Samuel Neves <samuel.c.p.neves@gmail.com>, Tomer Ashur <tomer.ashur@esat.kuleuven.be>, Martin Willi <martin@strongswan.org> Subject: [PATCH 9/17] zinc: Add x86 accelerated poly1305 Date: Fri, 22 Mar 2019 14:29:47 +0800 [thread overview] Message-ID: <E1h7DgZ-0001IW-E8@gondobar> (raw) In-Reply-To: 20190322062740.nrwfx2rvmt7lzotj@gondor.apana.org.au This patch exposes the crypto API x86 poly1305 implementation through zinc. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> --- lib/zinc/Kconfig | 1 lib/zinc/poly1305/poly1305-x86_64-glue.c | 68 +++++++++++++++++++++++++++++++ lib/zinc/poly1305/poly1305.c | 4 + 3 files changed, 73 insertions(+) diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig index 70d45b3288d2..34fa49beb37f 100644 --- a/lib/zinc/Kconfig +++ b/lib/zinc/Kconfig @@ -8,6 +8,7 @@ config ZINC_CHACHA20 config ZINC_POLY1305 tristate select CRYPTO_POLY1305 + select CRYPTO_POLY1305_X86_64 if ZINC_ARCH_X86_64 config ZINC_SELFTEST bool "Zinc cryptography library self-tests" diff --git a/lib/zinc/poly1305/poly1305-x86_64-glue.c b/lib/zinc/poly1305/poly1305-x86_64-glue.c new file mode 100644 index 000000000000..9e6dbd40911d --- /dev/null +++ b/lib/zinc/poly1305/poly1305-x86_64-glue.c @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved. + */ + +#include <asm/cpufeature.h> +#include <asm/processor.h> +#include <asm/intel-family.h> + +static bool poly1305_use_sse2 __ro_after_init; +static bool *const poly1305_nobs[] __initconst = { + &poly1305_use_sse2 }; + +static void __init poly1305_fpu_init(void) +{ + poly1305_use_sse2 = boot_cpu_has(X86_FEATURE_XMM2); +} + +struct poly1305_arch_internal { + struct poly1305_internal base; + struct poly1305_simd_xtra x; +}; + +static inline bool poly1305_init_arch(void *ctx, + const u8 key[POLY1305_KEY_SIZE]) +{ + struct poly1305_arch_internal *state = ctx; + + poly1305_simd_init(&state->x); + poly1305_core_setkey(&state->base.r, key); + return true; +} + +static inline bool poly1305_blocks_arch(void *ctx, const u8 *inp, + size_t len, const u32 padbit, + simd_context_t *simd_context) +{ + struct poly1305_arch_internal *state = ctx; + + /* SIMD disables preemption, so relax after processing each page. */ + BUILD_BUG_ON(PAGE_SIZE < POLY1305_BLOCK_SIZE || + PAGE_SIZE % POLY1305_BLOCK_SIZE); + + if (!poly1305_use_sse2 || !simd_use(simd_context) || padbit) + return false; + + for (;;) { + const size_t bytes = min_t(size_t, len, PAGE_SIZE); + + poly1305_simd_blocks(&state->base.h, &state->base.r, + &state->x, inp, bytes); + + len -= bytes; + if (!len) + break; + inp += bytes; + simd_relax(simd_context); + } + + return true; +} + +static inline bool poly1305_emit_arch(void *ctx, u8 mac[POLY1305_MAC_SIZE], + const u32 nonce[4], + simd_context_t *simd_context) +{ + return false; +} diff --git a/lib/zinc/poly1305/poly1305.c b/lib/zinc/poly1305/poly1305.c index 1546f5ac6d1c..c6a88136abb7 100644 --- a/lib/zinc/poly1305/poly1305.c +++ b/lib/zinc/poly1305/poly1305.c @@ -22,6 +22,9 @@ struct poly1305_internal { struct poly1305_state h; }; +#if defined(CONFIG_ZINC_ARCH_X86_64) +#include "poly1305-x86_64-glue.c" +#else static inline bool poly1305_init_arch(void *ctx, const u8 key[POLY1305_KEY_SIZE]) { @@ -43,6 +46,7 @@ static bool *const poly1305_nobs[] __initconst = { }; static void __init poly1305_fpu_init(void) { } +#endif static void poly1305_init_generic(void *ctx, const u8 key[16]) { _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-03-22 6:42 UTC|newest] Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-03-22 6:27 [PATCH 0/17] Add zinc using existing algorithm implementations Herbert Xu 2019-03-22 6:27 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 1/17] asm: simd context helper API Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 2/17] crypto: chacha20 - Export chacha20 functions without crypto API Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 3/17] zinc: introduce minimal cryptography library Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 4/17] zinc: Add generic C implementation of chacha20 and self-test Herbert Xu 2019-03-22 6:29 ` [PATCH 5/17] zinc: Add x86 accelerated ChaCha20 Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 6/17] zinc: Add arm accelerated chacha20 Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 7/17] crypto: poly1305 - Export core functions without crypto API Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 8/17] zinc: Add generic C implementation of poly1305 and self-test Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` Herbert Xu [this message] 2019-03-22 6:29 ` [PATCH 9/17] zinc: Add x86 accelerated poly1305 Herbert Xu 2019-03-22 6:29 ` [PATCH 10/17] zinc: ChaCha20Poly1305 construction and selftest Herbert Xu 2019-03-22 6:29 ` [PATCH 11/17] zinc: BLAKE2s generic C implementation " Herbert Xu 2019-03-22 6:29 ` [PATCH 12/17] zinc: BLAKE2s x86_64 implementation Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 13/17] zinc: Curve25519 generic C implementations and selftest Herbert Xu 2019-03-22 6:29 ` [PATCH 14/17] zinc: Curve25519 x86_64 implementation Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 15/17] zinc: import Bernstein and Schwabe's Curve25519 ARM implementation Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 16/17] zinc: " Herbert Xu 2019-03-22 6:29 ` [PATCH 17/17] security/keys: rewrite big_key crypto to use Zinc Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:41 ` [PATCH 0/17] Add zinc using existing algorithm implementations Jason A. Donenfeld 2019-03-22 6:41 ` Jason A. Donenfeld 2019-03-22 7:56 ` Ard Biesheuvel 2019-03-22 7:56 ` Ard Biesheuvel 2019-03-22 8:10 ` Jason A. Donenfeld 2019-03-22 8:10 ` Jason A. Donenfeld 2019-03-22 17:48 ` Linus Torvalds 2019-03-22 17:48 ` Linus Torvalds 2019-03-25 9:10 ` Pascal Van Leeuwen 2019-03-25 9:10 ` Pascal Van Leeuwen 2019-03-26 9:46 ` Riku Voipio 2019-03-26 9:46 ` Riku Voipio 2019-04-09 16:14 ` Pascal Van Leeuwen 2019-04-09 16:14 ` Pascal Van Leeuwen 2019-04-09 16:14 ` Pascal Van Leeuwen 2019-03-25 10:43 ` Ard Biesheuvel 2019-03-25 10:43 ` Ard Biesheuvel 2019-03-25 10:45 ` Jason A. Donenfeld 2019-03-25 10:45 ` Jason A. Donenfeld
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=E1h7DgZ-0001IW-E8@gondobar \ --to=herbert@gondor.apana.org.au \ --cc=Jason@zx2c4.com \ --cc=ard.biesheuvel@linaro.org \ --cc=davem@davemloft.net \ --cc=ebiggers@kernel.org \ --cc=gkaiser@google.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-crypto@vger.kernel.org \ --cc=linux-fscrypt@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=martin@strongswan.org \ --cc=paulcrowley@google.com \ --cc=samuel.c.p.neves@gmail.com \ --cc=tomer.ashur@esat.kuleuven.be \ --cc=torvalds@linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.