Re: pahole v1.24: FAILED: load BTF from vmlinux: Invalid argument

[Yonghong Song <yhs@fb.com>]

On 8/26/22 10:01 AM, Jiri Olsa wrote:
> On Fri, Aug 26, 2022 at 09:51:54AM -0700, Yonghong Song wrote:
>>
>>
>> On 8/26/22 6:52 AM, Jiri Olsa wrote:
>>> On Fri, Aug 26, 2022 at 05:59:44AM +0300, Vitaly Chikunov wrote:
>>>> On Fri, Aug 26, 2022 at 05:52:20AM +0300, Vitaly Chikunov wrote:
>>>>> Arnaldo,
>>>>>
>>>>> On Thu, Aug 25, 2022 at 08:16:20PM +0300, Vitaly Chikunov wrote:
>>>>>> On Thu, Aug 25, 2022 at 01:47:59PM -0300, Arnaldo Carvalho de Melo wrote:
>>>>>>> On Thu, Aug 25, 2022, 1:35 PM Vitaly Chikunov <vt@altlinux.org> wrote:
>>>>>>>>
>>>>>>>> I also noticed that after upgrading pahole to v1.24 kernel build (tested on
>>>>>>>> v5.18.19, v5.15.63, sorry for not testing on mainline) fails with:
>>>>>>>>
>>>>>>>>       BTFIDS  vmlinux
>>>>>>>>     + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
>>>>>>>>     FAILED: load BTF from vmlinux: Invalid argument
>>>>>>>>
>>>>>>>> Perhaps, .tmp_vmlinux.btf is generated incorrectly? Downgrading dwarves to
>>>>>>>> v1.23 resolves the issue.
>>>>>>>>
>>>>>>>
>>>>>>> Can you try this, from Martin Reboredo (Archlinux):
>>>>>>>
>>>>>>> Can you try a build of the kernel or the by passing the
>>>>>>> --skip_encoding_btf_enum64 to scripts/pahole-flags.sh?
>>>>>>>
>>>>>>> Here's a patch for either in tree scripts/pahole-flags.sh or
>>>>>>> /usr/lib/modules/5.19.3-arch1-1/build/scripts/pahole-flags.sh
>>>>>>
>>>>>> This patch helped and kernel builds successfully after applying it.
>>>>>> (Didn't notice this suggestion in release discussion thread.)
>>>>>
>>>>> Even thought it now compiles with this patch, it does not boot
>>>>> afterwards (in virtme-like env), witch such console messages:
>>>>
>>>> I'm talking here about 5.15.62. Yes, proposed patch does not apply there
>>>> (since there is no `scripts/pahole-flags.sh`), but I updated
>>>> `scripts/link-vmlinux.sh` with the similar `if` to append
>>>> `--skip_encoding_btf_enum64` which lets then compilation pass.
>>>>
>>>> Thanks,
>>>>
>>>>>
>>>>>     [    0.767649] Run /init as init process
>>>>>     [    0.770858] BPF:[593] ENUM perf_event_task_context
>>>>>     [    0.771262] BPF:size=4 vlen=4
>>>>>     [    0.771511] BPF:
>>>>>     [    0.771680] BPF:Invalid btf_info kind_flag
>>>>>     [    0.772016] BPF:
>>>
>>> I can see the same on 5.15, it looks like the libbpf change that
>>> pahole is compiled with is setting the type's kflag for values < 0:
>>> (which is the case for perf_event_task_context enum first value)
>>>
>>>     dffbbdc2d988 libbpf: Add enum64 parsing and new enum64 public API
>>>
>>> but IIUC kflag should stay zero for normal enum otherwise the btf meta
>>> verifier screams
>>
>> This is deliberate so we can have sign bit set properly for 32bit enum.
>> To avoid this behavior, the correct way is to turn off enum64 support
>> in pahole with flag --skip_encoding_btf_enum64.
> 
> I used that as well, it wouldn't compile without
> 
> the error is during the boot where the standard enum has kflag set

I just tried latest bpf-next, using pahole 1.24, with and without
--skip_encoding_btf_enum64. The following are BTF encoding
for enum perf_event_task_context.

enum perf_event_task_context {
         perf_invalid_context = -1,
         perf_hw_context = 0,
         perf_sw_context,
         perf_nr_task_contexts,
};

With --skip_encoding_btf_enum64:
[2285] ENUM 'perf_event_task_context' encoding=UNSIGNED size=4 vlen=4
         'perf_invalid_context' val=4294967295
         'perf_hw_context' val=0
         'perf_sw_context' val=1
         'perf_nr_task_contexts' val=2

Without --skip_encoding_btf_enum64:
[3786] ENUM 'perf_event_task_context' encoding=SIGNED size=4 vlen=4
         'perf_invalid_context' val=-1
         'perf_hw_context' val=0
         'perf_sw_context' val=1
         'perf_nr_task_contexts' val=2

encoding SIGNED means kflag = 1 and UNSIGNED is the default meaning
kflag = 0. So it looks okay to me. Could you try to use latest
bpftool to dump vmlinux BTF for your vmlinux binary?

Regarding the corresponding pahole enum64 support, we have
the following code,

         if (conf_load->skip_encoding_btf_enum64)
                 err = btf__add_enum_value(encoder->btf, name, 
(uint32_t)value);
         else if (etype->size > 32)
                 err = btf__add_enum64_value(encoder->btf, name, value);
         else
                 err = btf__add_enum_value(encoder->btf, name, value);

If skip_encoding_btf_enum64 is enabled, the value will be passed
as '(uint32_t)value', so '__s64 value' in the parameter should be
a unsigned value and 'if (value < 0) ...' should not be
triggered if skip_encoding_btf_enum64 is enabled.

Jiri, could you double check your build environment?

> 
> jirka
> 
>>
>>>
>>> if I compile pahole with the libbpf change below I can boot 5.15 kernel
>>> normally
>>>
>>> Yonghong, any idea?
>>>
>>> thanks,
>>> jirka
>>>
>>>
>>> ---
>>> diff --git a/src/btf.c b/src/btf.c
>>> index 2d14f1a52d7a..53d7516e4b89 100644
>>> --- a/src/btf.c
>>> +++ b/src/btf.c
>>> @@ -2151,10 +2151,6 @@ int btf__add_enum_value(struct btf *btf, const char *name, __s64 value)
>>>    	t = btf_last_type(btf);
>>>    	btf_type_inc_vlen(t);
>>> -	/* if negative value, set signedness to signed */
>>> -	if (value < 0)
>>> -		t->info = btf_type_info(btf_kind(t), btf_vlen(t), true);
>>> -
>>>    	btf->hdr->type_len += sz;
>>>    	btf->hdr->str_off += sz;
>>>    	return 0;