[PATCH] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[PATCH] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[Peter Zijlstra]

STACK_FRAME_NON_STANDARD results in inconsistent relocation types
depending on .c or .S usage:

  Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
  Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
  0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
  0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
  0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
  0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
  0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 include/linux/objtool.h |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

--- a/include/linux/objtool.h
+++ b/include/linux/objtool.h
@@ -137,7 +137,11 @@ struct unwind_hint {
 
 .macro STACK_FRAME_NON_STANDARD func:req
 	.pushsection .discard.func_stack_frame_non_standard, "aw"
-		.long \func - .
+#ifdef CONFIG_64BIT
+		.quad \func
+#else
+		.long \func
+#endif
 	.popsection
 .endm
 

Re: [PATCH] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[Peter Zijlstra]

On Fri, Apr 29, 2022 at 11:20:24AM +0200, Peter Zijlstra wrote:
> 
> STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> depending on .c or .S usage:
> 
>   Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
>   Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
>   0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
>   0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
>   0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
>   0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
>   0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0

So that weird .rodata entry is optprobe_template_func.

It being in .rodata also means it's not validated and there is no ORC
data generated, is that all intentional? The changelog for:

  877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")

doesn't really say anything useful about any of that :/

I also don't see any kprobe/optprobe hooks in unwind.h, so what happens
if we hit an optprobe?

Re: [PATCH] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[Josh Poimboeuf]

On Fri, Apr 29, 2022 at 02:00:44PM +0200, Peter Zijlstra wrote:
> On Fri, Apr 29, 2022 at 11:20:24AM +0200, Peter Zijlstra wrote:
> > 
> > STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> > depending on .c or .S usage:
> > 
> >   Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
> >   Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
> >   0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
> >   0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
> >   0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
> >   0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
> >   0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0
> 
> So that weird .rodata entry is optprobe_template_func.
> 
> It being in .rodata also means it's not validated and there is no ORC
> data generated, is that all intentional? The changelog for:
> 
>   877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")
> 
> doesn't really say anything useful about any of that :/
> 
> I also don't see any kprobe/optprobe hooks in unwind.h, so what happens
> if we hit an optprobe?

Same as for any other generated code, the unwinder will try to fall back
to frame pointers, and if that doesn't work, the unwind stops.

That commit didn't change anything since it was already not being
directly executed anyway, but rather used to generate code on the fly.

And before that commit it was being ignored by ORC anyway, thanks to
STACK_FRAME_NON_STANDARD.  Which can now be removed since this code is
now data and objtool will no longer try to understand it.

-- 
Josh

Re: [PATCH] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[Peter Zijlstra]

On Fri, Apr 29, 2022 at 03:56:36PM -0700, Josh Poimboeuf wrote:
> On Fri, Apr 29, 2022 at 02:00:44PM +0200, Peter Zijlstra wrote:
> > On Fri, Apr 29, 2022 at 11:20:24AM +0200, Peter Zijlstra wrote:
> > > 
> > > STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> > > depending on .c or .S usage:
> > > 
> > >   Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
> > >   Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
> > >   0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
> > >   0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
> > >   0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
> > >   0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
> > >   0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0
> > 
> > So that weird .rodata entry is optprobe_template_func.
> > 
> > It being in .rodata also means it's not validated and there is no ORC
> > data generated, is that all intentional? The changelog for:
> > 
> >   877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")
> > 
> > doesn't really say anything useful about any of that :/
> > 
> > I also don't see any kprobe/optprobe hooks in unwind.h, so what happens
> > if we hit an optprobe?
> 
> Same as for any other generated code, the unwinder will try to fall back
> to frame pointers, and if that doesn't work, the unwind stops.
> 
> That commit didn't change anything since it was already not being
> directly executed anyway, but rather used to generate code on the fly.
> 
> And before that commit it was being ignored by ORC anyway, thanks to
> STACK_FRAME_NON_STANDARD.  Which can now be removed since this code is
> now data and objtool will no longer try to understand it.

Right; but I suppose I'm wondering if we should fix this. It seems a
rather sub-optimal state of affairs.

Re: [PATCH] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[Josh Poimboeuf]

On Sat, Apr 30, 2022 at 01:44:00PM +0200, Peter Zijlstra wrote:
> > > I also don't see any kprobe/optprobe hooks in unwind.h, so what happens
> > > if we hit an optprobe?
> > 
> > Same as for any other generated code, the unwinder will try to fall back
> > to frame pointers, and if that doesn't work, the unwind stops.
> > 
> > That commit didn't change anything since it was already not being
> > directly executed anyway, but rather used to generate code on the fly.
> > 
> > And before that commit it was being ignored by ORC anyway, thanks to
> > STACK_FRAME_NON_STANDARD.  Which can now be removed since this code is
> > now data and objtool will no longer try to understand it.
> 
> Right; but I suppose I'm wondering if we should fix this. It seems a
> rather sub-optimal state of affairs.

Masami recently fixed some kprobes ORC issues but I don't know if this
one was fixed.

As to the whether it's worth fixing, I dunno.  There are trade offs.

Depends on how common the stack trace is -- I'm guessing not very, since
I've never seen a bug report -- and how important it is to get to full
ORC coverage.  If our goal is full coverage, we'd need a way for
generated code to add/remove ORC entries.

-- 
Josh

Re: [PATCH] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[Masami Hiramatsu]

On Mon, 2 May 2022 10:59:21 -0700
Josh Poimboeuf <jpoimboe@redhat.com> wrote:

> On Sat, Apr 30, 2022 at 01:44:00PM +0200, Peter Zijlstra wrote:
> > > > I also don't see any kprobe/optprobe hooks in unwind.h, so what happens
> > > > if we hit an optprobe?
> > > 
> > > Same as for any other generated code, the unwinder will try to fall back
> > > to frame pointers, and if that doesn't work, the unwind stops.
> > > 
> > > That commit didn't change anything since it was already not being
> > > directly executed anyway, but rather used to generate code on the fly.

Ah, OK. So ORC will not work on the dynamically generated trampoline code.
Can we generate ORC information entry dynamically?
(E.g. copying ORC data from the original code)

> > > 
> > > And before that commit it was being ignored by ORC anyway, thanks to
> > > STACK_FRAME_NON_STANDARD.  Which can now be removed since this code is
> > > now data and objtool will no longer try to understand it.
> > 
> > Right; but I suppose I'm wondering if we should fix this. It seems a
> > rather sub-optimal state of affairs.
> 
> Masami recently fixed some kprobes ORC issues but I don't know if this
> one was fixed.

I've fixed the kretprobe ORC unwinder issue. I need to check the optprobe
case too.

> 
> As to the whether it's worth fixing, I dunno.  There are trade offs.
> 
> Depends on how common the stack trace is -- I'm guessing not very, since
> I've never seen a bug report -- and how important it is to get to full
> ORC coverage.  If our goal is full coverage, we'd need a way for
> generated code to add/remove ORC entries.

Agreed, if I can copy the ORC entries for the original code to the entries
for generated code, I can fix it.

Thank you,


> 
> -- 
> Josh
> 


-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [PATCH] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[Masami Hiramatsu]

On Fri, 29 Apr 2022 14:00:44 +0200
Peter Zijlstra <peterz@infradead.org> wrote:

> On Fri, Apr 29, 2022 at 11:20:24AM +0200, Peter Zijlstra wrote:
> > 
> > STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> > depending on .c or .S usage:
> > 
> >   Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
> >   Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
> >   0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
> >   0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
> >   0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
> >   0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
> >   0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0
> 
> So that weird .rodata entry is optprobe_template_func.
> 
> It being in .rodata also means it's not validated and there is no ORC
> data generated, is that all intentional? The changelog for:
> 
>   877b145f0f47 ("x86/kprobes: Move trampoline code into RODATA")
> 
> doesn't really say anything useful about any of that :/

This commit was introduced just for reducing attack surface (the
trampoline code is NOT executed but just copied into trampoline
buffers), but if the ORC unwinder doesn't work correctly, please
revert it.
I think there is no functional change.

Thanks,

-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [PATCH] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[Josh Poimboeuf]

On Fri, Apr 29, 2022 at 11:20:24AM +0200, Peter Zijlstra wrote:
> 
> STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> depending on .c or .S usage:
> 
>   Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
>   Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
>   0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
>   0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
>   0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
>   0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
>   0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0
> 
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> ---
>  include/linux/objtool.h |    6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> --- a/include/linux/objtool.h
> +++ b/include/linux/objtool.h
> @@ -137,7 +137,11 @@ struct unwind_hint {
>  
>  .macro STACK_FRAME_NON_STANDARD func:req
>  	.pushsection .discard.func_stack_frame_non_standard, "aw"
> -		.long \func - .
> +#ifdef CONFIG_64BIT
> +		.quad \func
> +#else
> +		.long \func
> +#endif
>  	.popsection
>  .endm

Can use _ASM_PTR here, and objtool.h needs synced to tools.

-- 
Josh

[PATCH v2] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[Peter Zijlstra]

On Fri, Apr 29, 2022 at 01:13:25PM -0700, Josh Poimboeuf wrote:
> Can use _ASM_PTR here, and objtool.h needs synced to tools.

Here goes..

---
Subject: objtool: Fix STACK_FRAME_NON_STANDARD reloc type
From: Peter Zijlstra <peterz@infradead.org>
Date: Tue Apr 26 17:08:53 CEST 2022

STACK_FRAME_NON_STANDARD results in inconsistent relocation types
depending on .c or .S usage:

  Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
  Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
  0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
  0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
  0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
  0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
  0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 include/linux/objtool.h       |    2 +-
 tools/include/linux/objtool.h |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

--- a/include/linux/objtool.h
+++ b/include/linux/objtool.h
@@ -137,7 +137,7 @@ struct unwind_hint {
 
 .macro STACK_FRAME_NON_STANDARD func:req
 	.pushsection .discard.func_stack_frame_non_standard, "aw"
-		.long \func - .
+	_ASM_PTR \func
 	.popsection
 .endm
 
--- a/tools/include/linux/objtool.h
+++ b/tools/include/linux/objtool.h
@@ -137,7 +137,7 @@ struct unwind_hint {
 
 .macro STACK_FRAME_NON_STANDARD func:req
 	.pushsection .discard.func_stack_frame_non_standard, "aw"
-		.long \func - .
+	_ASM_PTR \func
 	.popsection
 .endm
 

Re: [PATCH v2] objtool: Fix STACK_FRAME_NON_STANDARD reloc type

[Josh Poimboeuf]

On Fri, Apr 29, 2022 at 11:22:35PM +0200, Peter Zijlstra wrote:
> On Fri, Apr 29, 2022 at 01:13:25PM -0700, Josh Poimboeuf wrote:
> > Can use _ASM_PTR here, and objtool.h needs synced to tools.
> 
> Here goes..
> 
> ---
> Subject: objtool: Fix STACK_FRAME_NON_STANDARD reloc type
> From: Peter Zijlstra <peterz@infradead.org>
> Date: Tue Apr 26 17:08:53 CEST 2022
> 
> STACK_FRAME_NON_STANDARD results in inconsistent relocation types
> depending on .c or .S usage:
> 
>   Relocation section '.rela.discard.func_stack_frame_non_standard' at offset 0x3c01090 contains 5 entries:
>   Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
>   0000000000000000  00020c2200000002 R_X86_64_PC32          0000000000047b40 do_suspend_lowlevel + 0
>   0000000000000008  0002461e00000001 R_X86_64_64            00000000000480a0 machine_real_restart + 0
>   0000000000000010  0000001400000001 R_X86_64_64            0000000000000000 .rodata + b3d4
>   0000000000000018  0002444600000002 R_X86_64_PC32          00000000000678a0 __efi64_thunk + 0
>   0000000000000020  0002659d00000001 R_X86_64_64            0000000000113160 __crash_kexec + 0
> 
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> ---
>  include/linux/objtool.h       |    2 +-
>  tools/include/linux/objtool.h |    2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> --- a/include/linux/objtool.h
> +++ b/include/linux/objtool.h
> @@ -137,7 +137,7 @@ struct unwind_hint {
>  
>  .macro STACK_FRAME_NON_STANDARD func:req
>  	.pushsection .discard.func_stack_frame_non_standard, "aw"
> -		.long \func - .
> +	_ASM_PTR \func
>  	.popsection
>  .endm

This file needs to include <asm/asm.h>, otherwise you get:

arch/x86/kernel/acpi/wakeup_64.S: Assembler messages:
arch/x86/kernel/acpi/wakeup_64.S:132: Error: no such instruction: `_asm_ptr do_suspend_lowlevel'


-- 
Josh