Re: [PATCH v3 16/16] objtool,x86: Rewrite retpoline thunk calls

From: Nathan Chancellor <nathan@kernel.org>
To: Peter Zijlstra <peterz@infradead.org>, Lukasz Majczak <lma@semihalf.com>
Cc: "Nick Desaulniers" <ndesaulniers@google.com>,
	"Fāng-ruì Sòng" <maskray@google.com>,
	"Josh Poimboeuf" <jpoimboe@redhat.com>,
	"Guenter Roeck" <groeck@google.com>,
	"Juergen Gross" <jgross@suse.com>,
	"Łukasz Bartosik" <lb@semihalf.com>,
	LKML <linux-kernel@vger.kernel.org>,
	mbenes@suse.com, "Radosław Biernacki" <rad@semihalf.com>,
	upstream@semihalf.com,
	"maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)"
	<x86@kernel.org>,
	clang-built-linux <clang-built-linux@googlegroups.com>,
	"Sami Tolvanen" <samitolvanen@google.com>
Subject: Re: [PATCH v3 16/16] objtool,x86: Rewrite retpoline thunk calls
Date: Wed, 9 Jun 2021 08:56:04 -0700	[thread overview]
Message-ID: <6caa3fa0-f71c-1a3f-b944-57b518645e74@kernel.org> (raw)
In-Reply-To: <20210609150804.GF68208@worktop.programming.kicks-ass.net>

On 6/9/2021 8:08 AM, Peter Zijlstra wrote:
> On Wed, Jun 09, 2021 at 02:23:28PM +0200, Lukasz Majczak wrote:
>> śr., 9 cze 2021 o 09:20 Peter Zijlstra <peterz@infradead.org> napisał(a):
>>>
>>> On Wed, Jun 09, 2021 at 09:11:18AM +0200, Lukasz Majczak wrote:
>>>
>>>> I'm sorry I was on vacation last week - do you still need the requested debugs?
>>>
>>> If the patch here:
>>>
>>>    https://lkml.kernel.org/r/YL3q1qFO9QIRL/BA@hirez.programming.kicks-ass.net
>>>
>>> does not fix things for you (don't think it actually will), then yes,
>>> please send me the information requested.
>>
>> Ok, it didn't help. Peter, Josh I have sent you a private email with
>> requested information.
> 
> OK, I think I've found it. Check this one:
> 
>   5d5:   0f 85 00 00 00 00       jne    5db <cpuidle_reflect+0x22>       5d7: R_X86_64_PLT32     __x86_indirect_thunk_r11-0x4
> 
> 
> +Relocation section '.rela.altinstructions' at offset 0 contains 14 entries:
> +    Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
> 
> +0000000000000018  0000000200000002 R_X86_64_PC32          0000000000000000 .text + 5d5
> +000000000000001c  0000009200000002 R_X86_64_PC32          0000000000000000 __x86_indirect_alt_call_r11 + 0
> 
> Apparently we get conditional branches to retpoline thunks and objtool
> completely messes that up. I'm betting this also explains the problems
> Nathan is having.

Yes, the below patch gets my kernel back to booting so it seems the root 
cause is the same.

> *groan*,.. not sure what to do about this, except return to having
> objtool generate code, which everybody hated on. For now I'll make it
> skip the conditional branches.
> 
> I wonder if the compiler will also generate conditional tail calls, and
> what that does with static_call... now I have to check all that.
> 
> ---

Tested-by: Nathan Chancellor <nathan@kernel.org>

> diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decode.c
> index 24295d39713b..523aa4157f80 100644
> --- a/tools/objtool/arch/x86/decode.c
> +++ b/tools/objtool/arch/x86/decode.c
> @@ -747,6 +747,10 @@ int arch_rewrite_retpolines(struct objtool_file *file)
>   
>   	list_for_each_entry(insn, &file->retpoline_call_list, call_node) {
>   
> +		if (insn->type != INSN_JUMP_DYNAMIC &&
> +		    insn->type != INSN_CALL_DYNAMIC)
> +			continue;
> +
>   		if (!strcmp(insn->sec->name, ".text.__x86.indirect_thunk"))
>   			continue;
>   
> 