From: Linus Torvalds <torvalds@linux-foundation.org>
To: Eric Dumazet <edumazet@google.com>
Cc: LKML <linux-kernel@vger.kernel.org>, netdev <netdev@vger.kernel.org>
Subject: Re: x86 copy performance regression
Date: Fri, 26 May 2023 10:17:03 -0700 [thread overview]
Message-ID: <CAHk-=whtDupvWtj_ow11wU4_u=KvifTqno=5mW1VofyehjdVRA@mail.gmail.com> (raw)
In-Reply-To: <CAHk-=whkci5ck5Him8Lx5ECKHEtj=bipYmOCGe8DWrrp8uDq5g@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 536 bytes --]
On Fri, May 26, 2023 at 10:00 AM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> Let me go look at it some more. I *really* didn't want to make the
> code worse for ERMS
Oh well. I'll think about it some more in the hope that I can come up
with something clever that doesn't make objtool hate me, but in the
meantime let me just give you the "not clever" patch.
It generates an annoying six-byte jump when the small 2-byte one would
work just fine, but I guess only my pride is wounded.
Linus
[-- Attachment #2: patch.diff --]
[-- Type: text/x-patch, Size: 877 bytes --]
arch/x86/lib/copy_user_64.S | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/x86/lib/copy_user_64.S b/arch/x86/lib/copy_user_64.S
index 4fc5c2de2de4..7e972224b0ba 100644
--- a/arch/x86/lib/copy_user_64.S
+++ b/arch/x86/lib/copy_user_64.S
@@ -7,6 +7,8 @@
*/
#include <linux/linkage.h>
+#include <asm/cpufeatures.h>
+#include <asm/alternative.h>
#include <asm/asm.h>
#include <asm/export.h>
@@ -29,7 +31,7 @@
*/
SYM_FUNC_START(rep_movs_alternative)
cmpq $64,%rcx
- jae .Lunrolled
+ alternative "jae .Lunrolled", "jae .Llarge", X86_FEATURE_ERMS
cmp $8,%ecx
jae .Lword
@@ -65,6 +67,12 @@ SYM_FUNC_START(rep_movs_alternative)
_ASM_EXTABLE_UA( 2b, .Lcopy_user_tail)
_ASM_EXTABLE_UA( 3b, .Lcopy_user_tail)
+.Llarge:
+0: rep movsb
+1: RET
+
+ _ASM_EXTABLE_UA( 0b, 1b)
+
.p2align 4
.Lunrolled:
10: movq (%rsi),%r8
next prev parent reply other threads:[~2023-05-26 17:17 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-26 15:00 x86 copy performance regression Eric Dumazet
2023-05-26 16:29 ` Linus Torvalds
2023-05-26 16:37 ` Eric Dumazet
2023-05-26 16:56 ` Linus Torvalds
2023-05-26 17:00 ` Linus Torvalds
2023-05-26 17:17 ` Linus Torvalds [this message]
2023-05-26 17:25 ` Eric Dumazet
2023-05-26 17:40 ` Linus Torvalds
2023-05-26 17:51 ` Eric Dumazet
2023-05-26 18:33 ` Linus Torvalds
2023-05-26 18:55 ` Eric Dumazet
2023-05-26 20:37 ` Linus Torvalds
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAHk-=whtDupvWtj_ow11wU4_u=KvifTqno=5mW1VofyehjdVRA@mail.gmail.com' \
--to=torvalds@linux-foundation.org \
--cc=edumazet@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).