From: Andy Lutomirski <luto@kernel.org>
To: x86@kernel.org
Cc: LKML <linux-kernel@vger.kernel.org>,
"Krzysztof Mazur" <krzysiek@podlesie.net>,
"Krzysztof Olędzki" <ole@ans.pl>, "Arnd Bergmann" <arnd@arndb.de>,
"Andy Lutomirski" <luto@kernel.org>
Subject: [PATCH v2 2/4] x86/mmx: Use KFPU_387 for MMX string operations
Date: Tue, 19 Jan 2021 09:39:00 -0800 [thread overview]
Message-ID: <cd68d61ece035201804a06e6993a2bd06bfe298b.1611077835.git.luto@kernel.org> (raw)
In-Reply-To: <cover.1611077835.git.luto@kernel.org>
The default kernel_fpu_begin() doesn't work on systems that support XMM but
haven't yet enabled CR4.OSFXSR. This causes crashes when _mmx_memcpy() is
called too early because LDMXCSR generates #UD when the aforementioned bit
is clear.
Fix it by using kernel_fpu_begin_mask(KFPU_387) explicitly.
Fixes: 7ad816762f9b ("x86/fpu: Reset MXCSR to default in kernel_fpu_begin()")
Reported-by: Krzysztof Mazur <krzysiek@podlesie.net>
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
arch/x86/lib/mmx_32.c | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)
diff --git a/arch/x86/lib/mmx_32.c b/arch/x86/lib/mmx_32.c
index 4321fa02e18d..2a6ad7aa148a 100644
--- a/arch/x86/lib/mmx_32.c
+++ b/arch/x86/lib/mmx_32.c
@@ -26,6 +26,16 @@
#include <asm/fpu/api.h>
#include <asm/asm.h>
+/*
+ * For MMX, we use KFPU_387. MMX instructions are not affected by MXCSR,
+ * but both AMD and Intel documentation states that even integer MMX
+ * operations will result in #MF if an exception is pending in FCW.
+ *
+ * We don't need EMMS afterwards because, after we call kernel_fpu_end(),
+ * any subsequent user of the 387 stack will reinitialize it using
+ * KFPU_387.
+ */
+
void *_mmx_memcpy(void *to, const void *from, size_t len)
{
void *p;
@@ -37,7 +47,7 @@ void *_mmx_memcpy(void *to, const void *from, size_t len)
p = to;
i = len >> 6; /* len/64 */
- kernel_fpu_begin();
+ kernel_fpu_begin_mask(KFPU_387);
__asm__ __volatile__ (
"1: prefetch (%0)\n" /* This set is 28 bytes */
@@ -127,7 +137,7 @@ static void fast_clear_page(void *page)
{
int i;
- kernel_fpu_begin();
+ kernel_fpu_begin_mask(KFPU_387);
__asm__ __volatile__ (
" pxor %%mm0, %%mm0\n" : :
@@ -160,7 +170,7 @@ static void fast_copy_page(void *to, void *from)
{
int i;
- kernel_fpu_begin();
+ kernel_fpu_begin_mask(KFPU_387);
/*
* maybe the prefetch stuff can go before the expensive fnsave...
@@ -247,7 +257,7 @@ static void fast_clear_page(void *page)
{
int i;
- kernel_fpu_begin();
+ kernel_fpu_begin_mask(KFPU_387);
__asm__ __volatile__ (
" pxor %%mm0, %%mm0\n" : :
@@ -282,7 +292,7 @@ static void fast_copy_page(void *to, void *from)
{
int i;
- kernel_fpu_begin();
+ kernel_fpu_begin_mask(KFPU_387);
__asm__ __volatile__ (
"1: prefetch (%0)\n"
--
2.29.2
next prev parent reply other threads:[~2021-01-19 18:56 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-19 17:38 [PATCH v2 0/4] x86/fpu: Reduce unnecessary FNINIT and MXCSR usage Andy Lutomirski
2021-01-19 17:38 ` [PATCH v2 1/4] x86/fpu: Add kernel_fpu_begin_mask() to selectively initialize state Andy Lutomirski
2021-01-19 19:34 ` Sean Christopherson
2021-01-20 11:53 ` Borislav Petkov
2021-01-20 12:19 ` Andy Lutomirski
2021-01-19 17:39 ` Andy Lutomirski [this message]
2021-01-20 18:18 ` [PATCH v2 2/4] x86/mmx: Use KFPU_387 for MMX string operations Borislav Petkov
2021-01-19 17:39 ` [PATCH v2 3/4] x86/fpu: Make the EFI FPU calling convention explicit Andy Lutomirski
2021-01-19 17:39 ` [PATCH v2 4/4] x86/fpu/64: Don't FNINIT in kernel_fpu_begin() Andy Lutomirski
2021-01-20 10:07 ` Peter Zijlstra
2021-01-20 18:28 ` Borislav Petkov
2021-01-20 7:51 ` [PATCH v2 0/4] x86/fpu: Reduce unnecessary FNINIT and MXCSR usage Krzysztof Olędzki
2021-01-21 5:03 ` Andy Lutomirski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cd68d61ece035201804a06e6993a2bd06bfe298b.1611077835.git.luto@kernel.org \
--to=luto@kernel.org \
--cc=arnd@arndb.de \
--cc=krzysiek@podlesie.net \
--cc=linux-kernel@vger.kernel.org \
--cc=ole@ans.pl \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).