From: Ard Biesheuvel <ardb@kernel.org> To: linux-arm-kernel@lists.infradead.org, linux@armlinux.org.uk Cc: linux-crypto@vger.kernel.org, Ard Biesheuvel <ardb@kernel.org>, Linus Walleij <linus.walleij@linaro.org>, Arnd Bergmann <arnd@arndb.de> Subject: [PATCH v2 2/2] ARM: permit non-nested kernel mode NEON in softirq context Date: Wed, 7 Dec 2022 11:39:36 +0100 [thread overview] Message-ID: <20221207103936.2198407-3-ardb@kernel.org> (raw) In-Reply-To: <20221207103936.2198407-1-ardb@kernel.org> We currently only permit kernel mode NEON in process context, to avoid the need to preserve/restore the NEON register file when taking an exception while running in the kernel. Like we did on arm64, we can relax this restriction substantially, by permitting kernel mode NEON from softirq context, while ensuring that softirq processing is disabled when the NEON is being used in task context. This guarantees that only NEON context belonging to user space needs to be preserved and restored, which is already taken care of. This is especially relevant for network encryption, where incoming frames are typically handled in softirq context, and deferring software decryption to a kernel thread or falling back to C code are both undesirable from a performance PoV. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> --- arch/arm/include/asm/simd.h | 8 ++++++++ arch/arm/vfp/vfpmodule.c | 13 ++++++------- 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/arch/arm/include/asm/simd.h b/arch/arm/include/asm/simd.h new file mode 100644 index 0000000000000000..82191dbd7e78a036 --- /dev/null +++ b/arch/arm/include/asm/simd.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include <linux/hardirq.h> + +static __must_check inline bool may_use_simd(void) +{ + return IS_ENABLED(CONFIG_KERNEL_MODE_NEON) && !in_hardirq(); +} diff --git a/arch/arm/vfp/vfpmodule.c b/arch/arm/vfp/vfpmodule.c index 8f5bc672b4aac04a..4e1a786df76df157 100644 --- a/arch/arm/vfp/vfpmodule.c +++ b/arch/arm/vfp/vfpmodule.c @@ -723,12 +723,12 @@ void kernel_neon_begin(void) local_bh_disable(); /* - * Kernel mode NEON is only allowed outside of interrupt context - * with preemption disabled. This will make sure that the kernel - * mode NEON register contents never need to be preserved. + * Kernel mode NEON is only allowed outside of hardirq context with + * preemption and softirq processing disabled. This will make sure that + * the kernel mode NEON register contents never need to be preserved. */ - BUG_ON(in_interrupt()); - cpu = get_cpu(); + BUG_ON(in_hardirq()); + cpu = __smp_processor_id(); fpexc = fmrx(FPEXC) | FPEXC_EN; fmxr(FPEXC, fpexc); @@ -744,7 +744,6 @@ void kernel_neon_begin(void) vfp_save_state(vfp_current_hw_state[cpu], fpexc); #endif vfp_current_hw_state[cpu] = NULL; - local_bh_enable(); } EXPORT_SYMBOL(kernel_neon_begin); @@ -752,7 +751,7 @@ void kernel_neon_end(void) { /* Disable the NEON/VFP unit. */ fmxr(FPEXC, fmrx(FPEXC) & ~FPEXC_EN); - put_cpu(); + local_bh_enable(); } EXPORT_SYMBOL(kernel_neon_end); -- 2.35.1
WARNING: multiple messages have this Message-ID (diff)
From: Ard Biesheuvel <ardb@kernel.org> To: linux-arm-kernel@lists.infradead.org, linux@armlinux.org.uk Cc: linux-crypto@vger.kernel.org, Ard Biesheuvel <ardb@kernel.org>, Linus Walleij <linus.walleij@linaro.org>, Arnd Bergmann <arnd@arndb.de> Subject: [PATCH v2 2/2] ARM: permit non-nested kernel mode NEON in softirq context Date: Wed, 7 Dec 2022 11:39:36 +0100 [thread overview] Message-ID: <20221207103936.2198407-3-ardb@kernel.org> (raw) In-Reply-To: <20221207103936.2198407-1-ardb@kernel.org> We currently only permit kernel mode NEON in process context, to avoid the need to preserve/restore the NEON register file when taking an exception while running in the kernel. Like we did on arm64, we can relax this restriction substantially, by permitting kernel mode NEON from softirq context, while ensuring that softirq processing is disabled when the NEON is being used in task context. This guarantees that only NEON context belonging to user space needs to be preserved and restored, which is already taken care of. This is especially relevant for network encryption, where incoming frames are typically handled in softirq context, and deferring software decryption to a kernel thread or falling back to C code are both undesirable from a performance PoV. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> --- arch/arm/include/asm/simd.h | 8 ++++++++ arch/arm/vfp/vfpmodule.c | 13 ++++++------- 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/arch/arm/include/asm/simd.h b/arch/arm/include/asm/simd.h new file mode 100644 index 0000000000000000..82191dbd7e78a036 --- /dev/null +++ b/arch/arm/include/asm/simd.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include <linux/hardirq.h> + +static __must_check inline bool may_use_simd(void) +{ + return IS_ENABLED(CONFIG_KERNEL_MODE_NEON) && !in_hardirq(); +} diff --git a/arch/arm/vfp/vfpmodule.c b/arch/arm/vfp/vfpmodule.c index 8f5bc672b4aac04a..4e1a786df76df157 100644 --- a/arch/arm/vfp/vfpmodule.c +++ b/arch/arm/vfp/vfpmodule.c @@ -723,12 +723,12 @@ void kernel_neon_begin(void) local_bh_disable(); /* - * Kernel mode NEON is only allowed outside of interrupt context - * with preemption disabled. This will make sure that the kernel - * mode NEON register contents never need to be preserved. + * Kernel mode NEON is only allowed outside of hardirq context with + * preemption and softirq processing disabled. This will make sure that + * the kernel mode NEON register contents never need to be preserved. */ - BUG_ON(in_interrupt()); - cpu = get_cpu(); + BUG_ON(in_hardirq()); + cpu = __smp_processor_id(); fpexc = fmrx(FPEXC) | FPEXC_EN; fmxr(FPEXC, fpexc); @@ -744,7 +744,6 @@ void kernel_neon_begin(void) vfp_save_state(vfp_current_hw_state[cpu], fpexc); #endif vfp_current_hw_state[cpu] = NULL; - local_bh_enable(); } EXPORT_SYMBOL(kernel_neon_begin); @@ -752,7 +751,7 @@ void kernel_neon_end(void) { /* Disable the NEON/VFP unit. */ fmxr(FPEXC, fmrx(FPEXC) & ~FPEXC_EN); - put_cpu(); + local_bh_enable(); } EXPORT_SYMBOL(kernel_neon_end); -- 2.35.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2022-12-07 10:39 UTC|newest] Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-12-07 10:39 [PATCH v2 0/2] ARM: allow kernel mode NEON in softirq context Ard Biesheuvel 2022-12-07 10:39 ` Ard Biesheuvel 2022-12-07 10:39 ` [PATCH v2 1/2] ARM: vfp: Manipulate VFP state with softirqs disabled Ard Biesheuvel 2022-12-07 10:39 ` Ard Biesheuvel 2022-12-15 10:22 ` Linus Walleij 2022-12-15 10:22 ` Linus Walleij 2022-12-07 10:39 ` Ard Biesheuvel [this message] 2022-12-07 10:39 ` [PATCH v2 2/2] ARM: permit non-nested kernel mode NEON in softirq context Ard Biesheuvel 2022-12-15 10:26 ` Linus Walleij 2022-12-15 10:26 ` Linus Walleij 2022-12-15 10:43 ` Ard Biesheuvel 2022-12-15 10:43 ` Ard Biesheuvel 2022-12-15 10:51 ` Russell King (Oracle) 2022-12-15 10:51 ` Russell King (Oracle) 2022-12-15 11:48 ` Ard Biesheuvel 2022-12-15 11:48 ` Ard Biesheuvel 2022-12-12 14:37 ` [PATCH v2 0/2] ARM: allow " Martin Willi 2022-12-12 14:37 ` Martin Willi 2022-12-13 16:56 ` Ard Biesheuvel 2022-12-13 16:56 ` Ard Biesheuvel
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20221207103936.2198407-3-ardb@kernel.org \ --to=ardb@kernel.org \ --cc=arnd@arndb.de \ --cc=linus.walleij@linaro.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-crypto@vger.kernel.org \ --cc=linux@armlinux.org.uk \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.