From: Ard Biesheuvel <firstname.lastname@example.org> To: email@example.com Cc: Ingo Molnar <firstname.lastname@example.org>, Herbert Xu <email@example.com>, Peter Zijlstra <firstname.lastname@example.org>, Catalin Marinas <email@example.com>, Sebastian Andrzej Siewior <firstname.lastname@example.org>, Dave Martin <email@example.com>, Eric Biggers <firstname.lastname@example.org>, Mark Brown <email@example.com>, Andy Lutomirski <firstname.lastname@example.org>, Thomas Gleixner <email@example.com>, Will Deacon <firstname.lastname@example.org>, Ard Biesheuvel <email@example.com>, firstname.lastname@example.org Subject: [PATCH v2 0/9] running kernel mode SIMD with softirqs disabled Date: Tue, 2 Mar 2021 10:01:09 +0100 [thread overview] Message-ID: <email@example.com> (raw) [ TL;DR for the non-ARM folks on CC: disabling softirq processing when using SIMD in kernel mode could reduce complexity and improve performance, but we need to decide whether we can do this, and how much softirq processing latency we can tolerate. If we can find a satisfactory solution for this, we might do the same for x86 and 32-bit ARM as well. However, based on preliminary off-list discussions with peterz and luto, it seems that for x86, there is a preference for using per-CPU buffers to preserve/restore the task context's kernel mode SIMD state when the task is interrupted to perform kernel mode SIMD in softirq context. On arm64, we actually had this arrangement before, and removed it because it made reasoning about preserving/restoring userland SVE state (32 SIMD registers of up to 2 kbit in size) rather complex. ] The crypto API provides two ways to invoke symmetric encryption algorithms: - synchronously, where the transformation is guaranteed to be done by the time the function returns; - asynchronously, where the function may return with a -EINPROGRESS return code, and a completion will be signalled when the transformation is done. The latter is mainly intended for h/w accelerators, where the throughput would be severely limited by the latency otherwise. However, it is also being used for software algorithms based on SIMD instructions, which cannot be issued from any context (the rules are not the same on each architecture, but typically, SIMD can be used in task context, or in softirq context if it was not taken while the SIMD was already in use in kernel mode). Many users of the crypto API exist in the kernel today that opt out of this asynchronous interface (802.11, macsec, kerberos, sw kTLS), or use a library interface which is fundamentally synchronous (wireguard). This means we end up using a degraded mode for the contended case (a scalar fallback) as well as the uncontended case (generic GCM/CCM/CTR chaining mode templates wrapped around the SIMD cipher as opposed to accelerated implementations of the full chaining modes in question). Note that scalar AES runs ~20x slower than the SIMD instruction based version. So let's address this for arm64, by reorganizing kernel mode SIMD support so that the SIMD unit can always be assumed to be available. This means we need to defer softirq processing when grabbing the NEON unit in task context, so that any use of it in softirq context is guaranteed not to interrupt any code that was already using the NEON. This obviously impacts softirq processing latency, which is why the existing conditional yield support is modified to take pending softirqs into account. Change since RFC/v1: - add patch to remove obsolete cond_yield_neon macros - rebased onto new, simplified cond_yield macro - include patches to remove the async path from all arm64 crypto skciphers and AEADs Previous RFC version:  https://firstname.lastname@example.org/ The first 3 patches will need to go through the arm64 tree, so once this series is reviewed, some coordination is required between the arm64 and crypto trees to get this merged without conflicts. Cc: Dave Martin <email@example.com> Cc: Mark Brown <firstname.lastname@example.org> Cc: Herbert Xu <email@example.com> Cc: Eric Biggers <firstname.lastname@example.org> Cc: Will Deacon <email@example.com> Cc: Catalin Marinas <firstname.lastname@example.org> Cc: Thomas Gleixner <email@example.com> Cc: Peter Zijlstra <firstname.lastname@example.org> Cc: Sebastian Andrzej Siewior <email@example.com> Cc: Ingo Molnar <firstname.lastname@example.org> Cc: Andy Lutomirski <email@example.com> Ard Biesheuvel (9): arm64: assembler: remove conditional NEON yield macros arm64: assembler: introduce wxN aliases for wN registers arm64: fpsimd: run kernel mode NEON with softirqs disabled crypto: aead - disallow en/decrypt for non-task or non-softirq context crypto: skcipher - disallow en/decrypt for non-task or non-softirq context crypto: arm64/gcm-aes-ce - remove non-SIMD fallback path crypto: arm64/aes-ccm - remove non-SIMD fallback path crypto: arm64/aes-ce - stop using SIMD helper for skciphers crypto: arm64/aes-neonbs - stop using SIMD helper for skciphers arch/arm64/crypto/Kconfig | 3 - arch/arm64/crypto/aes-ce-ccm-glue.c | 151 +++----------- arch/arm64/crypto/aes-glue.c | 102 ++-------- arch/arm64/crypto/aes-modes.S | 2 +- arch/arm64/crypto/aes-neonbs-glue.c | 122 +----------- arch/arm64/crypto/ghash-ce-glue.c | 209 +++++--------------- arch/arm64/crypto/sha1-ce-core.S | 2 +- arch/arm64/crypto/sha2-ce-core.S | 2 +- arch/arm64/crypto/sha3-ce-core.S | 4 +- arch/arm64/crypto/sha512-ce-core.S | 2 +- arch/arm64/include/asm/assembler.h | 106 +++------- arch/arm64/kernel/asm-offsets.c | 2 + arch/arm64/kernel/fpsimd.c | 4 +- crypto/aead.c | 10 + crypto/skcipher.c | 10 + 15 files changed, 162 insertions(+), 569 deletions(-) -- 2.30.1 _______________________________________________ linux-arm-kernel mailing list firstname.lastname@example.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next reply other threads:[~2021-03-02 9:03 UTC|newest] Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-03-02 9:01 Ard Biesheuvel [this message] 2021-03-02 9:01 ` [PATCH v2 1/9] arm64: assembler: remove conditional NEON yield macros Ard Biesheuvel 2021-03-30 9:52 ` Will Deacon 2021-04-12 8:39 ` Ard Biesheuvel 2021-03-02 9:01 ` [PATCH v2 2/9] arm64: assembler: introduce wxN aliases for wN registers Ard Biesheuvel 2021-03-30 9:59 ` Will Deacon 2021-03-02 9:01 ` [PATCH v2 3/9] arm64: fpsimd: run kernel mode NEON with softirqs disabled Ard Biesheuvel 2021-03-30 10:36 ` Will Deacon 2021-03-02 9:01 ` [PATCH v2 4/9] crypto: aead - disallow en/decrypt for non-task or non-softirq context Ard Biesheuvel 2021-03-02 9:01 ` [PATCH v2 5/9] crypto: skcipher " Ard Biesheuvel 2021-03-02 9:01 ` [PATCH v2 6/9] crypto: arm64/gcm-aes-ce - remove non-SIMD fallback path Ard Biesheuvel 2021-03-02 9:01 ` [PATCH v2 7/9] crypto: arm64/aes-ccm " Ard Biesheuvel 2021-03-02 9:01 ` [PATCH v2 8/9] crypto: arm64/aes-ce - stop using SIMD helper for skciphers Ard Biesheuvel 2021-03-02 9:01 ` [PATCH v2 9/9] crypto: arm64/aes-neonbs " Ard Biesheuvel 2021-04-12 13:11 ` (subset) [PATCH v2 0/9] running kernel mode SIMD with softirqs disabled Catalin Marinas
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [PATCH v2 0/9] running kernel mode SIMD with softirqs disabled' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).