From: Juergen Gross <jgross@suse.com>
To: Josh Poimboeuf <jpoimboe@redhat.com>,
Peter Zijlstra <peterz@infradead.org>
Cc: tglx@linutronix.de, linux-kernel@vger.kernel.org,
joro@8bytes.org, boris.ostrovsky@oracle.com, x86@kernel.org,
mbenes@suse.com, rostedt@goodmis.org, dvyukov@google.com,
elver@google.com
Subject: Re: [PATCH v2 01/24] x86/xen: Mark cpu_bringup_and_idle() as dead_end_function
Date: Mon, 30 Aug 2021 07:55:02 +0200 [thread overview]
Message-ID: <d27cd344-1323-803c-d26d-9aa851681b69@suse.com> (raw)
In-Reply-To: <fc90e972-f81f-3706-b13a-988b16264650@suse.com>
[-- Attachment #1.1.1: Type: text/plain, Size: 5462 bytes --]
On 23.08.21 10:40, Juergen Gross wrote:
> On 20.08.21 21:31, Josh Poimboeuf wrote:
>> On Fri, Aug 20, 2021 at 12:22:28PM -0700, Josh Poimboeuf wrote:
>>> On Thu, Jun 24, 2021 at 11:41:00AM +0200, Peter Zijlstra wrote:
>>>> The asm_cpu_bringup_and_idle() function is required to push the return
>>>> value on the stack in order to make ORC happy, but the only reason
>>>> objtool doesn't complain is because of a happy accident.
>>>>
>>>> The thing is that asm_cpu_bringup_and_idle() doesn't return, so
>>>> validate_branch() never terminates and falls through to the next
>>>> function, which in the normal case is the hypercall_page. And that, as
>>>> it happens, is 4095 NOPs and a RET.
>>>>
>>>> Make asm_cpu_bringup_and_idle() terminate on it's own, by making the
>>>> function it calls as a dead-end. This way we no longer rely on what
>>>> code happens to come after.
>>>>
>>>> Fixes: c3881eb58d56 ("x86/xen: Make the secondary CPU idle tasks
>>>> reliable")
>>>> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
>>>
>>> Looks right. Only problem is, with my assembler I get this:
>>>
>>> arch/x86/kernel/head_64.o: warning: objtool: .text+0x5:
>>> unreachable instruction
>>>
>>> Because gas insists on jumping over the page of nops...
>>>
>>> 0000000000000000 <asm_cpu_bringup_and_idle>:
>>> 0: e8 00 00 00 00 callq 5
>>> <asm_cpu_bringup_and_idle+0x5>
>>> 1: R_X86_64_PLT32 cpu_bringup_and_idle-0x4
>>> 5: e9 f6 0f 00 00 jmpq 1000
>>> <xen_hypercall_set_trap_table>
>>> a: 66 66 2e 0f 1f 84 00 data16 nopw %cs:0x0(%rax,%rax,1)
>>> 11: 00 00 00 00
>>> 15: 66 66 2e 0f 1f 84 00 data16 nopw %cs:0x0(%rax,%rax,1)
>>> 1c: 00 00 00 00
>>> 20: 66 66 2e 0f 1f 84 00 data16 nopw %cs:0x0(%rax,%rax,1)
>>> 27: 00 00 00 00
>>> 2b: 66 66 2e 0f 1f 84 00 data16 nopw %cs:0x0(%rax,%rax,1)
>>> 32: 00 00 00 00
>>> 36: 66 66 2e 0f 1f 84 00 data16 nopw %cs:0x0(%rax,%rax,1)
>>> 3d: 00 00 00 00
>>
>> Here's a fix:
>>
>> From: Josh Poimboeuf <jpoimboe@redhat.com>
>> Subject: [PATCH] x86/xen: Move hypercall_page to top of the file
>>
>> Because hypercall_page is page-aligned, the assembler inexplicably adds
>> an unreachable jump from after the end of the previous code to the
>> beginning of hypercall_page.
>>
>> That confuses objtool, understandably. It also creates significant text
>> fragmentation. As a result, much of the object file is wasted text
>> (nops).
>>
>> Move hypercall_page to the beginning of the file to both prevent the
>> text fragmentation and avoid the dead jump instruction.
>>
>> $ size /tmp/head_64.before.o /tmp/head_64.after.o
>> text data bss dec hex filename
>> 10924 307252 4096 322272 4eae0
>> /tmp/head_64.before.o
>> 6823 307252 4096 318171 4dadb
>> /tmp/head_64.after.o
>>
>> Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com
>
> Reviewed-by: Juergen Gross <jgross@suse.com>
Umm, will this be carried through the tip tree, or shall I take it in
the xen tree?
Juergen
>
>
> Juergen
>
>> ---
>> arch/x86/xen/xen-head.S | 34 +++++++++++++++++-----------------
>> 1 file changed, 17 insertions(+), 17 deletions(-)
>>
>> diff --git a/arch/x86/xen/xen-head.S b/arch/x86/xen/xen-head.S
>> index cb6538ae2fe0..488944d6d430 100644
>> --- a/arch/x86/xen/xen-head.S
>> +++ b/arch/x86/xen/xen-head.S
>> @@ -20,6 +20,23 @@
>> #include <xen/interface/xen-mca.h>
>> #include <asm/xen/interface.h>
>> +.pushsection .text
>> + .balign PAGE_SIZE
>> +SYM_CODE_START(hypercall_page)
>> + .rept (PAGE_SIZE / 32)
>> + UNWIND_HINT_FUNC
>> + .skip 31, 0x90
>> + ret
>> + .endr
>> +
>> +#define HYPERCALL(n) \
>> + .equ xen_hypercall_##n, hypercall_page + __HYPERVISOR_##n * 32; \
>> + .type xen_hypercall_##n, @function; .size xen_hypercall_##n, 32
>> +#include <asm/xen-hypercalls.h>
>> +#undef HYPERCALL
>> +SYM_CODE_END(hypercall_page)
>> +.popsection
>> +
>> #ifdef CONFIG_XEN_PV
>> __INIT
>> SYM_CODE_START(startup_xen)
>> @@ -64,23 +81,6 @@ SYM_CODE_END(asm_cpu_bringup_and_idle)
>> #endif
>> #endif
>> -.pushsection .text
>> - .balign PAGE_SIZE
>> -SYM_CODE_START(hypercall_page)
>> - .rept (PAGE_SIZE / 32)
>> - UNWIND_HINT_FUNC
>> - .skip 31, 0x90
>> - ret
>> - .endr
>> -
>> -#define HYPERCALL(n) \
>> - .equ xen_hypercall_##n, hypercall_page + __HYPERVISOR_##n * 32; \
>> - .type xen_hypercall_##n, @function; .size xen_hypercall_##n, 32
>> -#include <asm/xen-hypercalls.h>
>> -#undef HYPERCALL
>> -SYM_CODE_END(hypercall_page)
>> -.popsection
>> -
>> ELFNOTE(Xen, XEN_ELFNOTE_GUEST_OS, .asciz "linux")
>> ELFNOTE(Xen, XEN_ELFNOTE_GUEST_VERSION, .asciz "2.6")
>> ELFNOTE(Xen, XEN_ELFNOTE_XEN_VERSION, .asciz "xen-3.0")
>>
>
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
next prev parent reply other threads:[~2021-08-30 5:55 UTC|newest]
Thread overview: 101+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-24 9:40 [PATCH v2 00/24] objtool/x86: noinstr vs PARAVIRT Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 01/24] x86/xen: Mark cpu_bringup_and_idle() as dead_end_function Peter Zijlstra
2021-06-24 10:49 ` Juergen Gross
2021-06-30 11:47 ` Miroslav Benes
2021-08-20 19:22 ` Josh Poimboeuf
2021-08-20 19:31 ` Josh Poimboeuf
2021-08-23 8:40 ` Juergen Gross
2021-08-30 5:55 ` Juergen Gross [this message]
2021-08-30 7:48 ` Peter Zijlstra
2021-08-30 7:56 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] x86/xen: Move hypercall_page to top of the file tip-bot2 for Josh Poimboeuf
2021-09-15 15:49 ` [tip: objtool/core] x86/xen: Mark cpu_bringup_and_idle() as dead_end_function tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 02/24] objtool: Introduce CFI hash Peter Zijlstra
2021-06-30 12:45 ` Miroslav Benes
2021-07-01 8:35 ` Peter Zijlstra
2021-07-01 8:53 ` Miroslav Benes
2021-08-20 22:27 ` Josh Poimboeuf
2021-08-25 10:13 ` Peter Zijlstra
2021-08-25 12:49 ` Peter Zijlstra
2021-08-25 10:15 ` Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 03/24] objtool: Handle __sanitize_cov*() tail calls Peter Zijlstra
2021-06-25 11:05 ` Marco Elver
2021-06-28 9:07 ` Peter Zijlstra
2021-06-25 13:38 ` Steven Rostedt
2021-06-26 8:18 ` Peter Zijlstra
2021-06-29 1:07 ` Steven Rostedt
2021-08-20 23:17 ` Josh Poimboeuf
2021-08-25 10:19 ` Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 04/24] x86/kvm: Always inline sev_*guest() Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 05/24] x86/kvm: Always inline vmload() / vmsave() Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 06/24] x86: Always inline context_tracking_guest_enter() Peter Zijlstra
2021-08-20 23:22 ` Josh Poimboeuf
2021-08-20 23:34 ` Josh Poimboeuf
2021-08-25 12:16 ` Peter Zijlstra
[not found] ` <YSZxtBHNJzoUbqdE@ravnborg.org>
2021-08-26 4:10 ` Josh Poimboeuf
2021-08-25 13:38 ` Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 07/24] x86/kvm: Always inline to_svm() Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 08/24] x86/kvm: Always inline evmcs_write64() Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 09/24] x86: Always inline ip_within_syscall_gap() Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 10/24] x86/sev: Fix noinstr for vc_ghcb_invalidate() Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 11/24] locking/lockdep: Avoid RCU-induced noinstr fail Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 12/24] x86/paravirt: Mark arch_local_irq_*() __always_inline Peter Zijlstra
2021-06-24 10:51 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 13/24] x86/paravirt: Use PVOP_* for paravirt calls Peter Zijlstra
2021-06-24 10:52 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 14/24] x86/xen: Make read_cr2() noinstr Peter Zijlstra
2021-06-24 10:55 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 15/24] x86/xen: Make write_cr2() noinstr Peter Zijlstra
2021-06-24 10:56 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 16/24] x86/xen: Make get_debugreg() noinstr Peter Zijlstra
2021-06-24 10:57 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 17/24] x86/xen: Make set_debugreg() noinstr Peter Zijlstra
2021-06-24 10:59 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 18/24] x86/xen: Make save_fl() noinstr Peter Zijlstra
2021-06-24 11:07 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 19/24] x86/xen: Make hypercall_page noinstr Peter Zijlstra
2021-06-24 11:08 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 20/24] x86/xen: Make irq_enable() noinstr Peter Zijlstra
2021-06-24 11:09 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 21/24] x86/xen: Make irq_disable() noinstr Peter Zijlstra
2021-06-24 11:10 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 22/24] x86/xen: Mark xen_force_evtchn_callback() noinstr Peter Zijlstra
2021-06-24 11:11 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 23/24] x86/xen: Rework the xen_{cpu,irq,mmu}_ops[] arrays Peter Zijlstra
2021-06-24 12:12 ` Juergen Gross
2021-09-15 15:49 ` [tip: objtool/core] x86/xen: Rework the xen_{cpu,irq,mmu}_opsarrays tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 9:41 ` [PATCH v2 24/24] objtool: Support pv_opsindirect calls for noinstr Peter Zijlstra
2021-09-15 15:49 ` [tip: objtool/core] " tip-bot2 for Peter Zijlstra
2021-09-17 12:58 ` tip-bot2 for Peter Zijlstra
2021-06-24 13:05 ` [PATCH v2 00/24] objtool/x86: noinstr vs PARAVIRT Steven Rostedt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d27cd344-1323-803c-d26d-9aa851681b69@suse.com \
--to=jgross@suse.com \
--cc=boris.ostrovsky@oracle.com \
--cc=dvyukov@google.com \
--cc=elver@google.com \
--cc=joro@8bytes.org \
--cc=jpoimboe@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mbenes@suse.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).