From: Herbert Xu <herbert@gondor.apana.org.au> To: Linus Torvalds <torvalds@linux-foundation.org>, "David S. Miller" <davem@davemloft.net>, "Jason A. Donenfeld" <Jason@zx2c4.com>, Eric Biggers <ebiggers@kernel.org>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Linux Crypto Mailing List <linux-crypto@vger.kernel.org>, linux-fscrypt@vger.kernel.org, linux-arm-kernel@lists.infradead.org, LKML <linux-kernel@vger.kernel.org>, Paul Crowley <paulcrowley@google.com>, Greg Kaiser <gkaiser@google.com>, Samuel Neves <samuel.c.p.neves@gmail.com>, Tomer Ashur <tomer.ashur@esat.kuleuven.be>, Martin Willi <martin@strongswan.org> Subject: [PATCH 6/17] zinc: Add arm accelerated chacha20 Date: Fri, 22 Mar 2019 14:29:43 +0800 [thread overview] Message-ID: <E1h7DgV-0001Hv-7y@gondobar> (raw) In-Reply-To: 20190322062740.nrwfx2rvmt7lzotj@gondor.apana.org.au This patch exposes the crypto API arm neon chacha20 implementation through zinc. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> --- lib/zinc/Kconfig | 2 + lib/zinc/chacha20/chacha20-arm-glue.c | 68 ++++++++++++++++++++++++++++++++++ lib/zinc/chacha20/chacha20.c | 2 + 3 files changed, 72 insertions(+) diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig index 010547fa6c9d..2d1750c50919 100644 --- a/lib/zinc/Kconfig +++ b/lib/zinc/Kconfig @@ -2,6 +2,8 @@ config ZINC_CHACHA20 tristate select CRYPTO_CHACHA20 select CRYPTO_CHACHA20_X86_64 if ZINC_ARCH_X86_64 + select CRYPTO_CHACHA20_NEON if ZINC_ARCH_ARM + select CRYPTO_CHACHA20_NEON if ZINC_ARCH_ARM64 config ZINC_SELFTEST bool "Zinc cryptography library self-tests" diff --git a/lib/zinc/chacha20/chacha20-arm-glue.c b/lib/zinc/chacha20/chacha20-arm-glue.c new file mode 100644 index 000000000000..2b6c88cf516b --- /dev/null +++ b/lib/zinc/chacha20/chacha20-arm-glue.c @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved. + */ + +#include <asm/hwcap.h> +#include <asm/neon.h> +#if defined(CONFIG_ZINC_ARCH_ARM) +#include <asm/system_info.h> +#include <asm/cputype.h> +#endif +#include <crypto/chacha.h> + +static bool chacha20_use_neon __ro_after_init; +static bool *const chacha20_nobs[] __initconst = { &chacha20_use_neon }; +static void __init chacha20_fpu_init(void) +{ +#if defined(CONFIG_ZINC_ARCH_ARM64) + chacha20_use_neon = elf_hwcap & HWCAP_ASIMD; +#elif defined(CONFIG_ZINC_ARCH_ARM) + switch (read_cpuid_part()) { + case ARM_CPU_PART_CORTEX_A7: + case ARM_CPU_PART_CORTEX_A5: + /* The Cortex-A7 and Cortex-A5 do not perform well with the NEON + * implementation but do incredibly with the scalar one and use + * less power. + */ + break; + default: + chacha20_use_neon = elf_hwcap & HWCAP_NEON; + } +#endif +} + +static inline bool chacha20_arch(struct chacha20_ctx *ctx, u8 *dst, + const u8 *src, size_t len, + simd_context_t *simd_context) +{ + /* SIMD disables preemption, so relax after processing each page. */ + BUILD_BUG_ON(PAGE_SIZE < CHACHA20_BLOCK_SIZE || + PAGE_SIZE % CHACHA20_BLOCK_SIZE); + + if (!IS_ENABLED(CONFIG_KERNEL_MODE_NEON) || !chacha20_use_neon || + len < CHACHA20_BLOCK_SIZE * 3 || !simd_use(simd_context)) { + + for (;;) { + const size_t bytes = min_t(size_t, len, PAGE_SIZE); + + crypto_chacha_neon(ctx->state, dst, src, bytes, 20); + + len -= bytes; + if (!len) + break; + dst += bytes; + src += bytes; + simd_relax(simd_context); + } + + return true; +} + +static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], + const u8 nonce[HCHACHA20_NONCE_SIZE], + const u8 key[HCHACHA20_KEY_SIZE], + simd_context_t *simd_context) +{ + return false; +} diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c index c84fe504623f..314525f86757 100644 --- a/lib/zinc/chacha20/chacha20.c +++ b/lib/zinc/chacha20/chacha20.c @@ -19,6 +19,8 @@ #if defined(CONFIG_ZINC_ARCH_X86_64) #include "chacha20-x86_64-glue.c" +#elif defined(CONFIG_ZINC_ARCH_ARM) || defined(CONFIG_ZINC_ARCH_ARM64) +#include "chacha20-arm-glue.c" #else static bool *const chacha20_nobs[] __initconst = { }; static void __init chacha20_fpu_init(void)
WARNING: multiple messages have this Message-ID (diff)
From: Herbert Xu <herbert@gondor.apana.org.au> To: Linus Torvalds <torvalds@linux-foundation.org>, "David S. Miller" <davem@davemloft.net>, "Jason A. Donenfeld" <Jason@zx2c4.com>, Eric Biggers <ebiggers@kernel.org>, Ard Biesheuvel <ard.biesheuvel@linaro.org>, Linux Crypto Mailing List <linux-crypto@vger.kernel.org>, linux-fscrypt@vger.kernel.org, linux-arm-kernel@lists.infradead.org, LKML <linux-kernel@vger.kernel.org>, Paul Crowley <paulcrowley@google.com>, Greg Kaiser <gkaiser@google.com>, Samuel Neves <samuel.c.p.neves@gmail.com>, Tomer Ashur <tomer.ashur@esat.kuleuven.be>, Martin Willi <martin@strongswan.org> Subject: [PATCH 6/17] zinc: Add arm accelerated chacha20 Date: Fri, 22 Mar 2019 14:29:43 +0800 [thread overview] Message-ID: <E1h7DgV-0001Hv-7y@gondobar> (raw) In-Reply-To: 20190322062740.nrwfx2rvmt7lzotj@gondor.apana.org.au This patch exposes the crypto API arm neon chacha20 implementation through zinc. Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> --- lib/zinc/Kconfig | 2 + lib/zinc/chacha20/chacha20-arm-glue.c | 68 ++++++++++++++++++++++++++++++++++ lib/zinc/chacha20/chacha20.c | 2 + 3 files changed, 72 insertions(+) diff --git a/lib/zinc/Kconfig b/lib/zinc/Kconfig index 010547fa6c9d..2d1750c50919 100644 --- a/lib/zinc/Kconfig +++ b/lib/zinc/Kconfig @@ -2,6 +2,8 @@ config ZINC_CHACHA20 tristate select CRYPTO_CHACHA20 select CRYPTO_CHACHA20_X86_64 if ZINC_ARCH_X86_64 + select CRYPTO_CHACHA20_NEON if ZINC_ARCH_ARM + select CRYPTO_CHACHA20_NEON if ZINC_ARCH_ARM64 config ZINC_SELFTEST bool "Zinc cryptography library self-tests" diff --git a/lib/zinc/chacha20/chacha20-arm-glue.c b/lib/zinc/chacha20/chacha20-arm-glue.c new file mode 100644 index 000000000000..2b6c88cf516b --- /dev/null +++ b/lib/zinc/chacha20/chacha20-arm-glue.c @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Copyright (C) 2015-2018 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved. + */ + +#include <asm/hwcap.h> +#include <asm/neon.h> +#if defined(CONFIG_ZINC_ARCH_ARM) +#include <asm/system_info.h> +#include <asm/cputype.h> +#endif +#include <crypto/chacha.h> + +static bool chacha20_use_neon __ro_after_init; +static bool *const chacha20_nobs[] __initconst = { &chacha20_use_neon }; +static void __init chacha20_fpu_init(void) +{ +#if defined(CONFIG_ZINC_ARCH_ARM64) + chacha20_use_neon = elf_hwcap & HWCAP_ASIMD; +#elif defined(CONFIG_ZINC_ARCH_ARM) + switch (read_cpuid_part()) { + case ARM_CPU_PART_CORTEX_A7: + case ARM_CPU_PART_CORTEX_A5: + /* The Cortex-A7 and Cortex-A5 do not perform well with the NEON + * implementation but do incredibly with the scalar one and use + * less power. + */ + break; + default: + chacha20_use_neon = elf_hwcap & HWCAP_NEON; + } +#endif +} + +static inline bool chacha20_arch(struct chacha20_ctx *ctx, u8 *dst, + const u8 *src, size_t len, + simd_context_t *simd_context) +{ + /* SIMD disables preemption, so relax after processing each page. */ + BUILD_BUG_ON(PAGE_SIZE < CHACHA20_BLOCK_SIZE || + PAGE_SIZE % CHACHA20_BLOCK_SIZE); + + if (!IS_ENABLED(CONFIG_KERNEL_MODE_NEON) || !chacha20_use_neon || + len < CHACHA20_BLOCK_SIZE * 3 || !simd_use(simd_context)) { + + for (;;) { + const size_t bytes = min_t(size_t, len, PAGE_SIZE); + + crypto_chacha_neon(ctx->state, dst, src, bytes, 20); + + len -= bytes; + if (!len) + break; + dst += bytes; + src += bytes; + simd_relax(simd_context); + } + + return true; +} + +static inline bool hchacha20_arch(u32 derived_key[CHACHA20_KEY_WORDS], + const u8 nonce[HCHACHA20_NONCE_SIZE], + const u8 key[HCHACHA20_KEY_SIZE], + simd_context_t *simd_context) +{ + return false; +} diff --git a/lib/zinc/chacha20/chacha20.c b/lib/zinc/chacha20/chacha20.c index c84fe504623f..314525f86757 100644 --- a/lib/zinc/chacha20/chacha20.c +++ b/lib/zinc/chacha20/chacha20.c @@ -19,6 +19,8 @@ #if defined(CONFIG_ZINC_ARCH_X86_64) #include "chacha20-x86_64-glue.c" +#elif defined(CONFIG_ZINC_ARCH_ARM) || defined(CONFIG_ZINC_ARCH_ARM64) +#include "chacha20-arm-glue.c" #else static bool *const chacha20_nobs[] __initconst = { }; static void __init chacha20_fpu_init(void) _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-03-22 6:31 UTC|newest] Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-03-22 6:27 [PATCH 0/17] Add zinc using existing algorithm implementations Herbert Xu 2019-03-22 6:27 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 1/17] asm: simd context helper API Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 2/17] crypto: chacha20 - Export chacha20 functions without crypto API Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 3/17] zinc: introduce minimal cryptography library Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 4/17] zinc: Add generic C implementation of chacha20 and self-test Herbert Xu 2019-03-22 6:29 ` [PATCH 5/17] zinc: Add x86 accelerated ChaCha20 Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` Herbert Xu [this message] 2019-03-22 6:29 ` [PATCH 6/17] zinc: Add arm accelerated chacha20 Herbert Xu 2019-03-22 6:29 ` [PATCH 7/17] crypto: poly1305 - Export core functions without crypto API Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 8/17] zinc: Add generic C implementation of poly1305 and self-test Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 9/17] zinc: Add x86 accelerated poly1305 Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 10/17] zinc: ChaCha20Poly1305 construction and selftest Herbert Xu 2019-03-22 6:29 ` [PATCH 11/17] zinc: BLAKE2s generic C implementation " Herbert Xu 2019-03-22 6:29 ` [PATCH 12/17] zinc: BLAKE2s x86_64 implementation Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 13/17] zinc: Curve25519 generic C implementations and selftest Herbert Xu 2019-03-22 6:29 ` [PATCH 14/17] zinc: Curve25519 x86_64 implementation Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 15/17] zinc: import Bernstein and Schwabe's Curve25519 ARM implementation Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:29 ` [PATCH 16/17] zinc: " Herbert Xu 2019-03-22 6:29 ` [PATCH 17/17] security/keys: rewrite big_key crypto to use Zinc Herbert Xu 2019-03-22 6:29 ` Herbert Xu 2019-03-22 6:41 ` [PATCH 0/17] Add zinc using existing algorithm implementations Jason A. Donenfeld 2019-03-22 6:41 ` Jason A. Donenfeld 2019-03-22 7:56 ` Ard Biesheuvel 2019-03-22 7:56 ` Ard Biesheuvel 2019-03-22 8:10 ` Jason A. Donenfeld 2019-03-22 8:10 ` Jason A. Donenfeld 2019-03-22 17:48 ` Linus Torvalds 2019-03-22 17:48 ` Linus Torvalds 2019-03-25 9:10 ` Pascal Van Leeuwen 2019-03-25 9:10 ` Pascal Van Leeuwen 2019-03-26 9:46 ` Riku Voipio 2019-03-26 9:46 ` Riku Voipio 2019-04-09 16:14 ` Pascal Van Leeuwen 2019-04-09 16:14 ` Pascal Van Leeuwen 2019-04-09 16:14 ` Pascal Van Leeuwen 2019-03-25 10:43 ` Ard Biesheuvel 2019-03-25 10:43 ` Ard Biesheuvel 2019-03-25 10:45 ` Jason A. Donenfeld 2019-03-25 10:45 ` Jason A. Donenfeld
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=E1h7DgV-0001Hv-7y@gondobar \ --to=herbert@gondor.apana.org.au \ --cc=Jason@zx2c4.com \ --cc=ard.biesheuvel@linaro.org \ --cc=davem@davemloft.net \ --cc=ebiggers@kernel.org \ --cc=gkaiser@google.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-crypto@vger.kernel.org \ --cc=linux-fscrypt@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=martin@strongswan.org \ --cc=paulcrowley@google.com \ --cc=samuel.c.p.neves@gmail.com \ --cc=tomer.ashur@esat.kuleuven.be \ --cc=torvalds@linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.