From: "tip-bot2 for Peter Zijlstra" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>,
Josh Poimboeuf <jpoimboe@redhat.com>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: [tip: x86/core] x86/word-at-a-time: Remove .fixup usage
Date: Mon, 13 Dec 2021 09:50:09 -0000 [thread overview]
Message-ID: <163938900955.23020.14868045137660023155.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20211110101326.141775772@infradead.org>
The following commit has been merged into the x86/core branch of tip:
Commit-ID: b7760780257354bb14de62abed868405b844fa13
Gitweb: https://git.kernel.org/tip/b7760780257354bb14de62abed868405b844fa13
Author: Peter Zijlstra <peterz@infradead.org>
AuthorDate: Wed, 10 Nov 2021 11:01:23 +01:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Sat, 11 Dec 2021 09:09:50 +01:00
x86/word-at-a-time: Remove .fixup usage
Rewrite load_unaligned_zeropad() to not require .fixup text.
This is easiest done using asm-goto-output, where we can stick a C
label in the exception table entry. The fallback version isn't nearly
so nice but should work.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com>
Link: https://lore.kernel.org/r/20211110101326.141775772@infradead.org
---
arch/x86/include/asm/word-at-a-time.h | 66 ++++++++++++++++++--------
1 file changed, 47 insertions(+), 19 deletions(-)
diff --git a/arch/x86/include/asm/word-at-a-time.h b/arch/x86/include/asm/word-at-a-time.h
index 06006b0..8338b04 100644
--- a/arch/x86/include/asm/word-at-a-time.h
+++ b/arch/x86/include/asm/word-at-a-time.h
@@ -77,30 +77,58 @@ static inline unsigned long find_zero(unsigned long mask)
* and the next page not being mapped, take the exception and
* return zeroes in the non-existing part.
*/
+#ifdef CONFIG_CC_HAS_ASM_GOTO_OUTPUT
+
static inline unsigned long load_unaligned_zeropad(const void *addr)
{
- unsigned long ret, dummy;
+ unsigned long offset, data;
+ unsigned long ret;
+
+ asm_volatile_goto(
+ "1: mov %[mem], %[ret]\n"
+
+ _ASM_EXTABLE(1b, %l[do_exception])
+
+ : [ret] "=r" (ret)
+ : [mem] "m" (*(unsigned long *)addr)
+ : : do_exception);
+
+ return ret;
+
+do_exception:
+ offset = (unsigned long)addr & (sizeof(long) - 1);
+ addr = (void *)((unsigned long)addr & ~(sizeof(long) - 1));
+ data = *(unsigned long *)addr;
+ ret = data >> offset * 8;
+
+ return ret;
+}
- asm(
- "1:\tmov %2,%0\n"
+#else /* !CONFIG_CC_HAS_ASM_GOTO_OUTPUT */
+
+static inline unsigned long load_unaligned_zeropad(const void *addr)
+{
+ unsigned long offset, data;
+ unsigned long ret, err = 0;
+
+ asm( "1: mov %[mem], %[ret]\n"
"2:\n"
- ".section .fixup,\"ax\"\n"
- "3:\t"
- "lea %2,%1\n\t"
- "and %3,%1\n\t"
- "mov (%1),%0\n\t"
- "leal %2,%%ecx\n\t"
- "andl %4,%%ecx\n\t"
- "shll $3,%%ecx\n\t"
- "shr %%cl,%0\n\t"
- "jmp 2b\n"
- ".previous\n"
- _ASM_EXTABLE(1b, 3b)
- :"=&r" (ret),"=&c" (dummy)
- :"m" (*(unsigned long *)addr),
- "i" (-sizeof(unsigned long)),
- "i" (sizeof(unsigned long)-1));
+
+ _ASM_EXTABLE_FAULT(1b, 2b)
+
+ : [ret] "=&r" (ret), "+a" (err)
+ : [mem] "m" (*(unsigned long *)addr));
+
+ if (unlikely(err)) {
+ offset = (unsigned long)addr & (sizeof(long) - 1);
+ addr = (void *)((unsigned long)addr & ~(sizeof(long) - 1));
+ data = *(unsigned long *)addr;
+ ret = data >> offset * 8;
+ }
+
return ret;
}
+#endif /* CONFIG_CC_HAS_ASM_GOTO_OUTPUT */
+
#endif /* _ASM_WORD_AT_A_TIME_H */
next prev parent reply other threads:[~2021-12-13 10:15 UTC|newest]
Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-10 10:01 [PATCH v2 00/23] x86: Remove anonymous out-of-line fixups Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 01/23] bitfield.h: Fix "type of reg too small for mask" test Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 02/23] x86,mmx_32: Remove .fixup usage Peter Zijlstra
2021-11-15 16:54 ` [PATCH v2.1 02/23] x86/mmx_32: Remove X86_USE_3DNOW Peter Zijlstra
2021-11-16 18:25 ` Borislav Petkov
2021-12-13 9:50 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 03/23] x86,copy_user_64: Remove .fixup usage Peter Zijlstra
2021-11-16 19:04 ` Borislav Petkov
2021-12-13 9:50 ` [tip: x86/core] x86/copy_user_64: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 04/23] x86,copy_mc_64: " Peter Zijlstra
2021-11-17 16:28 ` Borislav Petkov
2021-12-13 9:50 ` [tip: x86/core] x86/copy_mc_64: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 05/23] x86,entry_64: " Peter Zijlstra
2021-11-17 16:29 ` Borislav Petkov
2021-11-19 8:00 ` Lai Jiangshan
2021-12-13 9:50 ` [tip: x86/core] x86/entry_64: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 06/23] x86,entry_32: " Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/entry_32: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 07/23] x86,extable: Extend extable functionality Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/extable: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 08/23] x86,msr: Remove .fixup usage Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/msr: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 09/23] x86,futex: " Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/futex: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 10/23] x86,uaccess: " Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/uaccess: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 11/23] x86,xen: " Peter Zijlstra
2021-11-10 10:35 ` Juergen Gross
2021-11-10 11:00 ` Peter Zijlstra
2021-11-10 12:46 ` Peter Zijlstra
2021-11-10 12:52 ` Juergen Gross
2021-11-10 16:17 ` Peter Zijlstra
2021-11-11 7:20 ` Juergen Gross
2021-11-10 12:45 ` [PATCH v2.1 " Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/xen: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 12/23] x86,fpu: " Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/fpu: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 13/23] x86,segment: " Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/segment: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 14/23] x86,kvm: " Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/kvm: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 15/23] x86,vmx: " Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/vmx: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 16/23] x86,vmx: Provide asm-goto-output vmread Peter Zijlstra
2021-12-13 10:07 ` Peter Zijlstra
2021-12-14 18:44 ` Sean Christopherson
2021-12-14 22:46 ` Peter Zijlstra
2021-12-14 23:07 ` Sean Christopherson
2021-11-10 10:01 ` [PATCH v2 17/23] x86,checksum_32: Remove .fixup usage Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/checksum_32: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 18/23] x86,sgx: " Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/sgx: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 19/23] x86,usercopy_32: Simplify __copy_user_intel_nocache() Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/usercopy_32: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 20/23] x86,usercopy: Remove .fixup usage Peter Zijlstra
2021-11-11 7:51 ` Josh Poimboeuf
2021-11-11 8:28 ` Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] x86/usercopy: " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 21/23] x86,word-at-a-time: " Peter Zijlstra
2021-12-13 9:50 ` tip-bot2 for Peter Zijlstra [this message]
2021-11-10 10:01 ` [PATCH v2 22/23] x86: Remove .fixup section Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2021-11-10 10:01 ` [PATCH v2 23/23] objtool: Remove .fixup handling Peter Zijlstra
2021-12-13 9:50 ` [tip: x86/core] " tip-bot2 for Peter Zijlstra
2021-11-11 7:55 ` [PATCH v2 00/23] x86: Remove anonymous out-of-line fixups Josh Poimboeuf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=163938900955.23020.14868045137660023155.tip-bot2@tip-bot2 \
--to=tip-bot2@linutronix.de \
--cc=jpoimboe@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).