libbpf error: unknown register name 'r0' in asm

libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

Pulling the latest changes of libbpf and compiling my application with it,
I see the following error:

../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
unknown register name 'r0' in asm
                     : "r0", "r1", "r2", "r3", "r4", "r5");

The commit which introduced this change is:
80c7838600d39891f274e2f7508b95a75e4227c1

I'm not sure if I'm doing something wrong (missing include?), or this
is a genuine error

Yaniv

Re: libbpf error: unknown register name 'r0' in asm

[Daniel Borkmann]

[ Cc +Yonghong ]

On 10/9/20 6:05 PM, Yaniv Agman wrote:
> Pulling the latest changes of libbpf and compiling my application with it,
> I see the following error:
> 
> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> unknown register name 'r0' in asm
>                       : "r0", "r1", "r2", "r3", "r4", "r5");
> 
> The commit which introduced this change is:
> 80c7838600d39891f274e2f7508b95a75e4227c1
> 
> I'm not sure if I'm doing something wrong (missing include?), or this
> is a genuine error

Seems like your clang/llvm version might be too old.

Yonghong, do you happen to know from which version onwards there is proper support
for bpf inline asm? We could potentially wrap this around like this:

diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
index 2bdb7d6dbad2..0d6abc91bfc6 100644
--- a/tools/lib/bpf/bpf_helpers.h
+++ b/tools/lib/bpf/bpf_helpers.h
@@ -72,6 +72,7 @@
  /*
   * Helper function to perform a tail call with a constant/immediate map slot.
   */
+#if __clang_major__ >= 10
  static __always_inline void
  bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
  {
@@ -98,6 +99,7 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
                      :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
                      : "r0", "r1", "r2", "r3", "r4", "r5");
  }
+#endif /* __clang_major__ >= 10 */

  /*
   * Helper structure used by eBPF C program



> Yaniv
> 

Re: libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
<‪daniel@iogearbox.net‬‏>:‬
>
> [ Cc +Yonghong ]
>
> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> > Pulling the latest changes of libbpf and compiling my application with it,
> > I see the following error:
> >
> > ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> > unknown register name 'r0' in asm
> >                       : "r0", "r1", "r2", "r3", "r4", "r5");
> >
> > The commit which introduced this change is:
> > 80c7838600d39891f274e2f7508b95a75e4227c1
> >
> > I'm not sure if I'm doing something wrong (missing include?), or this
> > is a genuine error
>
> Seems like your clang/llvm version might be too old.

I'm using clang 10.0.1

>
> Yonghong, do you happen to know from which version onwards there is proper support
> for bpf inline asm? We could potentially wrap this around like this:
>
> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
> index 2bdb7d6dbad2..0d6abc91bfc6 100644
> --- a/tools/lib/bpf/bpf_helpers.h
> +++ b/tools/lib/bpf/bpf_helpers.h
> @@ -72,6 +72,7 @@
>   /*
>    * Helper function to perform a tail call with a constant/immediate map slot.
>    */
> +#if __clang_major__ >= 10
>   static __always_inline void
>   bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>   {
> @@ -98,6 +99,7 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>                       :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
>                       : "r0", "r1", "r2", "r3", "r4", "r5");
>   }
> +#endif /* __clang_major__ >= 10 */
>
>   /*
>    * Helper structure used by eBPF C program
>
>
>
> > Yaniv
> >
>

Re: libbpf error: unknown register name 'r0' in asm

[Daniel Borkmann]

On 10/9/20 6:56 PM, Yaniv Agman wrote:
> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> <‪daniel@iogearbox.net‬‏>:‬
>>
>> [ Cc +Yonghong ]
>>
>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
>>> Pulling the latest changes of libbpf and compiling my application with it,
>>> I see the following error:
>>>
>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
>>> unknown register name 'r0' in asm
>>>                        : "r0", "r1", "r2", "r3", "r4", "r5");
>>>
>>> The commit which introduced this change is:
>>> 80c7838600d39891f274e2f7508b95a75e4227c1
>>>
>>> I'm not sure if I'm doing something wrong (missing include?), or this
>>> is a genuine error
>>
>> Seems like your clang/llvm version might be too old.
> 
> I'm using clang 10.0.1

Ah, okay, I see. Would this diff do the trick for you?

diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
index 2bdb7d6dbad2..31e356831fcf 100644
--- a/tools/lib/bpf/bpf_helpers.h
+++ b/tools/lib/bpf/bpf_helpers.h
@@ -72,6 +72,7 @@
  /*
   * Helper function to perform a tail call with a constant/immediate map slot.
   */
+#if __clang_major__ >= 10 && defined(__bpf__)
  static __always_inline void
  bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
  {
@@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
  		     :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
  		     : "r0", "r1", "r2", "r3", "r4", "r5");
  }
+#else
+# define bpf_tail_call_static	bpf_tail_call
+#endif /* __clang_major__ >= 10 && __bpf__ */

  /*
   * Helper structure used by eBPF C program

Re: libbpf error: unknown register name 'r0' in asm

[Yonghong Song]

On 10/9/20 9:27 AM, Daniel Borkmann wrote:
> [ Cc +Yonghong ]
> 
> On 10/9/20 6:05 PM, Yaniv Agman wrote:
>> Pulling the latest changes of libbpf and compiling my application with 
>> it,
>> I see the following error:
>>
>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
>> unknown register name 'r0' in asm
>>                       : "r0", "r1", "r2", "r3", "r4", "r5");
>>
>> The commit which introduced this change is:
>> 80c7838600d39891f274e2f7508b95a75e4227c1
>>
>> I'm not sure if I'm doing something wrong (missing include?), or this
>> is a genuine error
> 
> Seems like your clang/llvm version might be too old.
> 
> Yonghong, do you happen to know from which version onwards there is 
> proper support
> for bpf inline asm? We could potentially wrap this around like this:

llvm6 starts to support inline asm.

> 
> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
> index 2bdb7d6dbad2..0d6abc91bfc6 100644
> --- a/tools/lib/bpf/bpf_helpers.h
> +++ b/tools/lib/bpf/bpf_helpers.h
> @@ -72,6 +72,7 @@
>   /*
>    * Helper function to perform a tail call with a constant/immediate 
> map slot.
>    */
> +#if __clang_major__ >= 10

Just change to __clang_major__ >= 6 should be okay, you may want to
double check though.

>   static __always_inline void
>   bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>   {
> @@ -98,6 +99,7 @@ bpf_tail_call_static(void *ctx, const void *map, const 
> __u32 slot)
>                       :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
>                       : "r0", "r1", "r2", "r3", "r4", "r5");
>   }
> +#endif /* __clang_major__ >= 10 */
> 
>   /*
>    * Helper structure used by eBPF C program
> 
> 
> 
>> Yaniv
>>
> 

Re: libbpf error: unknown register name 'r0' in asm

[Andrii Nakryiko]

On Fri, Oct 9, 2020 at 9:06 AM Yaniv Agman <yanivagman@gmail.com> wrote:
>
> Pulling the latest changes of libbpf and compiling my application with it,
> I see the following error:
>
> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> unknown register name 'r0' in asm
>                      : "r0", "r1", "r2", "r3", "r4", "r5");

are you including bpf_helpers.h from user-space code? You are using
recent enough Clang to not run into this problem for -target bpf, so
this seems like you are including bpf_helpers.h in the context where
it's not supposed to be included.


>
> The commit which introduced this change is:
> 80c7838600d39891f274e2f7508b95a75e4227c1
>
> I'm not sure if I'm doing something wrong (missing include?), or this
> is a genuine error
>
> Yaniv

Re: libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
<‪daniel@iogearbox.net‬‏>:‬
>
> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> > <‪daniel@iogearbox.net‬‏>:‬
> >>
> >> [ Cc +Yonghong ]
> >>
> >> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> >>> Pulling the latest changes of libbpf and compiling my application with it,
> >>> I see the following error:
> >>>
> >>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> >>> unknown register name 'r0' in asm
> >>>                        : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>
> >>> The commit which introduced this change is:
> >>> 80c7838600d39891f274e2f7508b95a75e4227c1
> >>>
> >>> I'm not sure if I'm doing something wrong (missing include?), or this
> >>> is a genuine error
> >>
> >> Seems like your clang/llvm version might be too old.
> >
> > I'm using clang 10.0.1
>
> Ah, okay, I see. Would this diff do the trick for you?

Yes! Now it compiles without any problems!

>
> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
> index 2bdb7d6dbad2..31e356831fcf 100644
> --- a/tools/lib/bpf/bpf_helpers.h
> +++ b/tools/lib/bpf/bpf_helpers.h
> @@ -72,6 +72,7 @@
>   /*
>    * Helper function to perform a tail call with a constant/immediate map slot.
>    */
> +#if __clang_major__ >= 10 && defined(__bpf__)
>   static __always_inline void
>   bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>   {
> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>                      :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
>                      : "r0", "r1", "r2", "r3", "r4", "r5");
>   }
> +#else
> +# define bpf_tail_call_static  bpf_tail_call
> +#endif /* __clang_major__ >= 10 && __bpf__ */
>
>   /*
>    * Helper structure used by eBPF C program

Re: libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:03 מאת ‪Andrii Nakryiko‬‏
<‪andrii.nakryiko@gmail.com‬‏>:‬
>
> On Fri, Oct 9, 2020 at 9:06 AM Yaniv Agman <yanivagman@gmail.com> wrote:
> >
> > Pulling the latest changes of libbpf and compiling my application with it,
> > I see the following error:
> >
> > ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> > unknown register name 'r0' in asm
> >                      : "r0", "r1", "r2", "r3", "r4", "r5");
>
> are you including bpf_helpers.h from user-space code? You are using
> recent enough Clang to not run into this problem for -target bpf, so
> this seems like you are including bpf_helpers.h in the context where
> it's not supposed to be included.
>

This happens when I compile the kernel bpf code, not for user-space code

>
> >
> > The commit which introduced this change is:
> > 80c7838600d39891f274e2f7508b95a75e4227c1
> >
> > I'm not sure if I'm doing something wrong (missing include?), or this
> > is a genuine error
> >
> > Yaniv

Re: libbpf error: unknown register name 'r0' in asm

[Daniel Borkmann]

On 10/9/20 8:09 PM, Yaniv Agman wrote:
> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> <‪daniel@iogearbox.net‬‏>:‬
>>
>> On 10/9/20 6:56 PM, Yaniv Agman wrote:
>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
>>> <‪daniel@iogearbox.net‬‏>:‬
>>>>
>>>> [ Cc +Yonghong ]
>>>>
>>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
>>>>> Pulling the latest changes of libbpf and compiling my application with it,
>>>>> I see the following error:
>>>>>
>>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
>>>>> unknown register name 'r0' in asm
>>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
>>>>>
>>>>> The commit which introduced this change is:
>>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
>>>>>
>>>>> I'm not sure if I'm doing something wrong (missing include?), or this
>>>>> is a genuine error
>>>>
>>>> Seems like your clang/llvm version might be too old.
>>>
>>> I'm using clang 10.0.1
>>
>> Ah, okay, I see. Would this diff do the trick for you?
> 
> Yes! Now it compiles without any problems!

Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.

>> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
>> index 2bdb7d6dbad2..31e356831fcf 100644
>> --- a/tools/lib/bpf/bpf_helpers.h
>> +++ b/tools/lib/bpf/bpf_helpers.h
>> @@ -72,6 +72,7 @@
>>    /*
>>     * Helper function to perform a tail call with a constant/immediate map slot.
>>     */
>> +#if __clang_major__ >= 10 && defined(__bpf__)
>>    static __always_inline void
>>    bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>>    {
>> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>>                       :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
>>                       : "r0", "r1", "r2", "r3", "r4", "r5");
>>    }
>> +#else
>> +# define bpf_tail_call_static  bpf_tail_call
>> +#endif /* __clang_major__ >= 10 && __bpf__ */
>>
>>    /*
>>     * Helper structure used by eBPF C program

Re: libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:21 מאת ‪Daniel Borkmann‬‏
<‪daniel@iogearbox.net‬‏>:‬
>
> On 10/9/20 8:09 PM, Yaniv Agman wrote:
> > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> > <‪daniel@iogearbox.net‬‏>:‬
> >>
> >> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> >>> <‪daniel@iogearbox.net‬‏>:‬
> >>>>
> >>>> [ Cc +Yonghong ]
> >>>>
> >>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> >>>>> Pulling the latest changes of libbpf and compiling my application with it,
> >>>>> I see the following error:
> >>>>>
> >>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> >>>>> unknown register name 'r0' in asm
> >>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>>>
> >>>>> The commit which introduced this change is:
> >>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> >>>>>
> >>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> >>>>> is a genuine error
> >>>>
> >>>> Seems like your clang/llvm version might be too old.
> >>>
> >>> I'm using clang 10.0.1
> >>
> >> Ah, okay, I see. Would this diff do the trick for you?
> >
> > Yes! Now it compiles without any problems!
>
> Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
>

Thanks!
Does this happen because I'm first compiling using "emit-llvm" and
then using llc?
I wish I could use bpf target directly, but I'm then having problems
with includes of asm code (like pt_regs and other stuff)

> >> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
> >> index 2bdb7d6dbad2..31e356831fcf 100644
> >> --- a/tools/lib/bpf/bpf_helpers.h
> >> +++ b/tools/lib/bpf/bpf_helpers.h
> >> @@ -72,6 +72,7 @@
> >>    /*
> >>     * Helper function to perform a tail call with a constant/immediate map slot.
> >>     */
> >> +#if __clang_major__ >= 10 && defined(__bpf__)
> >>    static __always_inline void
> >>    bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>    {
> >> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>                       :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
> >>                       : "r0", "r1", "r2", "r3", "r4", "r5");
> >>    }
> >> +#else
> >> +# define bpf_tail_call_static  bpf_tail_call
> >> +#endif /* __clang_major__ >= 10 && __bpf__ */
> >>
> >>    /*
> >>     * Helper structure used by eBPF C program
>

Re: libbpf error: unknown register name 'r0' in asm

[Andrii Nakryiko]

On Fri, Oct 9, 2020 at 11:21 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On 10/9/20 8:09 PM, Yaniv Agman wrote:
> > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> > <‪daniel@iogearbox.net‬‏>:‬
> >>
> >> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> >>> <‪daniel@iogearbox.net‬‏>:‬
> >>>>
> >>>> [ Cc +Yonghong ]
> >>>>
> >>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> >>>>> Pulling the latest changes of libbpf and compiling my application with it,
> >>>>> I see the following error:
> >>>>>
> >>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> >>>>> unknown register name 'r0' in asm
> >>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>>>
> >>>>> The commit which introduced this change is:
> >>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> >>>>>
> >>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> >>>>> is a genuine error
> >>>>
> >>>> Seems like your clang/llvm version might be too old.
> >>>
> >>> I'm using clang 10.0.1
> >>
> >> Ah, okay, I see. Would this diff do the trick for you?
> >
> > Yes! Now it compiles without any problems!
>
> Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
>

Am I the only one confused here?... Yonghong said it should be
supported as early as clang 6, Yaniv is using Clang 10 and is still
getting this error. Let's figure out what's the problem before adding
unnecessary checks.

I think it's not the clang_major check that helped, rather __bpf__
check. So please hold off on the fix, let's get to the bottom of this
first.

> >> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
> >> index 2bdb7d6dbad2..31e356831fcf 100644
> >> --- a/tools/lib/bpf/bpf_helpers.h
> >> +++ b/tools/lib/bpf/bpf_helpers.h
> >> @@ -72,6 +72,7 @@
> >>    /*
> >>     * Helper function to perform a tail call with a constant/immediate map slot.
> >>     */
> >> +#if __clang_major__ >= 10 && defined(__bpf__)
> >>    static __always_inline void
> >>    bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>    {
> >> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>                       :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
> >>                       : "r0", "r1", "r2", "r3", "r4", "r5");
> >>    }
> >> +#else
> >> +# define bpf_tail_call_static  bpf_tail_call
> >> +#endif /* __clang_major__ >= 10 && __bpf__ */
> >>
> >>    /*
> >>     * Helper structure used by eBPF C program
>

Re: libbpf error: unknown register name 'r0' in asm

[Andrii Nakryiko]

On Fri, Oct 9, 2020 at 11:33 AM Yaniv Agman <yanivagman@gmail.com> wrote:
>
> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:21 מאת ‪Daniel Borkmann‬‏
> <‪daniel@iogearbox.net‬‏>:‬
> >
> > On 10/9/20 8:09 PM, Yaniv Agman wrote:
> > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> > > <‪daniel@iogearbox.net‬‏>:‬
> > >>
> > >> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> > >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> > >>> <‪daniel@iogearbox.net‬‏>:‬
> > >>>>
> > >>>> [ Cc +Yonghong ]
> > >>>>
> > >>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> > >>>>> Pulling the latest changes of libbpf and compiling my application with it,
> > >>>>> I see the following error:
> > >>>>>
> > >>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> > >>>>> unknown register name 'r0' in asm
> > >>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
> > >>>>>
> > >>>>> The commit which introduced this change is:
> > >>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> > >>>>>
> > >>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> > >>>>> is a genuine error
> > >>>>
> > >>>> Seems like your clang/llvm version might be too old.
> > >>>
> > >>> I'm using clang 10.0.1
> > >>
> > >> Ah, okay, I see. Would this diff do the trick for you?
> > >
> > > Yes! Now it compiles without any problems!
> >
> > Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> >
>
> Thanks!
> Does this happen because I'm first compiling using "emit-llvm" and
> then using llc?

So this must be the reason, but I'll wait for Yonghong to confirm.

> I wish I could use bpf target directly, but I'm then having problems
> with includes of asm code (like pt_regs and other stuff)

Are you developing for a 32-bit platform? Or what exactly is the
problem? I've been trying to solve problems for 32-bit arches recently
by making libbpf smarter, that relies on CO-RE though. Is CO-RE an
option for you?

[...]

Re: libbpf error: unknown register name 'r0' in asm

[Daniel Borkmann]

On 10/9/20 8:35 PM, Andrii Nakryiko wrote:
> On Fri, Oct 9, 2020 at 11:21 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>> On 10/9/20 8:09 PM, Yaniv Agman wrote:
>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
>>> <‪daniel@iogearbox.net‬‏>:‬
>>>>
>>>> On 10/9/20 6:56 PM, Yaniv Agman wrote:
>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
>>>>> <‪daniel@iogearbox.net‬‏>:‬
>>>>>>
>>>>>> [ Cc +Yonghong ]
>>>>>>
>>>>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
>>>>>>> Pulling the latest changes of libbpf and compiling my application with it,
>>>>>>> I see the following error:
>>>>>>>
>>>>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
>>>>>>> unknown register name 'r0' in asm
>>>>>>>                          : "r0", "r1", "r2", "r3", "r4", "r5");
>>>>>>>
>>>>>>> The commit which introduced this change is:
>>>>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
>>>>>>>
>>>>>>> I'm not sure if I'm doing something wrong (missing include?), or this
>>>>>>> is a genuine error
>>>>>>
>>>>>> Seems like your clang/llvm version might be too old.
>>>>>
>>>>> I'm using clang 10.0.1
>>>>
>>>> Ah, okay, I see. Would this diff do the trick for you?
>>>
>>> Yes! Now it compiles without any problems!
>>
>> Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> 
> Am I the only one confused here?... Yonghong said it should be
> supported as early as clang 6, Yaniv is using Clang 10 and is still
> getting this error. Let's figure out what's the problem before adding
> unnecessary checks.
> 
> I think it's not the clang_major check that helped, rather __bpf__
> check. So please hold off on the fix, let's get to the bottom of this
> first.

I don't see confusion here (maybe other than which minimal clang/llvm version
libbpf should support). If we do `#if __clang_major__ >= 6 && defined(__bpf__)`
for the final patch, then this means that user passed clang -target bpf and
the min supported version for inline assembly was there, otherwise we fall back
to bpf_tail_call. In Yaniv's case, he probably had native target with -emit-llvm
and then used llc invocation.

>>>> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
>>>> index 2bdb7d6dbad2..31e356831fcf 100644
>>>> --- a/tools/lib/bpf/bpf_helpers.h
>>>> +++ b/tools/lib/bpf/bpf_helpers.h
>>>> @@ -72,6 +72,7 @@
>>>>     /*
>>>>      * Helper function to perform a tail call with a constant/immediate map slot.
>>>>      */
>>>> +#if __clang_major__ >= 10 && defined(__bpf__)
>>>>     static __always_inline void
>>>>     bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>>>>     {
>>>> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>>>>                        :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
>>>>                        : "r0", "r1", "r2", "r3", "r4", "r5");
>>>>     }
>>>> +#else
>>>> +# define bpf_tail_call_static  bpf_tail_call
>>>> +#endif /* __clang_major__ >= 10 && __bpf__ */
>>>>
>>>>     /*
>>>>      * Helper structure used by eBPF C program
>>

Re: libbpf error: unknown register name 'r0' in asm

[Andrii Nakryiko]

On Fri, Oct 9, 2020 at 11:41 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On 10/9/20 8:35 PM, Andrii Nakryiko wrote:
> > On Fri, Oct 9, 2020 at 11:21 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >> On 10/9/20 8:09 PM, Yaniv Agman wrote:
> >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> >>> <‪daniel@iogearbox.net‬‏>:‬
> >>>>
> >>>> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> >>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> >>>>> <‪daniel@iogearbox.net‬‏>:‬
> >>>>>>
> >>>>>> [ Cc +Yonghong ]
> >>>>>>
> >>>>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> >>>>>>> Pulling the latest changes of libbpf and compiling my application with it,
> >>>>>>> I see the following error:
> >>>>>>>
> >>>>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> >>>>>>> unknown register name 'r0' in asm
> >>>>>>>                          : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>>>>>
> >>>>>>> The commit which introduced this change is:
> >>>>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> >>>>>>>
> >>>>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> >>>>>>> is a genuine error
> >>>>>>
> >>>>>> Seems like your clang/llvm version might be too old.
> >>>>>
> >>>>> I'm using clang 10.0.1
> >>>>
> >>>> Ah, okay, I see. Would this diff do the trick for you?
> >>>
> >>> Yes! Now it compiles without any problems!
> >>
> >> Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> >
> > Am I the only one confused here?... Yonghong said it should be
> > supported as early as clang 6, Yaniv is using Clang 10 and is still
> > getting this error. Let's figure out what's the problem before adding
> > unnecessary checks.
> >
> > I think it's not the clang_major check that helped, rather __bpf__
> > check. So please hold off on the fix, let's get to the bottom of this
> > first.
>
> I don't see confusion here (maybe other than which minimal clang/llvm version
> libbpf should support). If we do `#if __clang_major__ >= 6 && defined(__bpf__)`
> for the final patch, then this means that user passed clang -target bpf and
> the min supported version for inline assembly was there, otherwise we fall back
> to bpf_tail_call. In Yaniv's case, he probably had native target with -emit-llvm
> and then used llc invocation.

The "-emit-llvm" was the part that we were missing and had to figure
it out, before we could discuss the fix.

>
> >>>> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
> >>>> index 2bdb7d6dbad2..31e356831fcf 100644
> >>>> --- a/tools/lib/bpf/bpf_helpers.h
> >>>> +++ b/tools/lib/bpf/bpf_helpers.h
> >>>> @@ -72,6 +72,7 @@
> >>>>     /*
> >>>>      * Helper function to perform a tail call with a constant/immediate map slot.
> >>>>      */
> >>>> +#if __clang_major__ >= 10 && defined(__bpf__)
> >>>>     static __always_inline void
> >>>>     bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>>>     {
> >>>> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>>>                        :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
> >>>>                        : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>>     }
> >>>> +#else
> >>>> +# define bpf_tail_call_static  bpf_tail_call

bpf_tail_call_static has very specific guarantees, so in cases where
we can't use inline assembly to satisfy those guarantees, I think we
should not just silently redefine bpf_tail_call_static as
bpf_tail_call, rather make compilation fail if someone is attempting
to use bpf_tail_call_static. _Static_assert could be used to provide a
better error message here, probably.

> >>>> +#endif /* __clang_major__ >= 10 && __bpf__ */
> >>>>
> >>>>     /*
> >>>>      * Helper structure used by eBPF C program
> >>
>

Re: libbpf error: unknown register name 'r0' in asm

[Yonghong Song]

On 10/9/20 11:59 AM, Andrii Nakryiko wrote:
> On Fri, Oct 9, 2020 at 11:41 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>>
>> On 10/9/20 8:35 PM, Andrii Nakryiko wrote:
>>> On Fri, Oct 9, 2020 at 11:21 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>>>> On 10/9/20 8:09 PM, Yaniv Agman wrote:
>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
>>>>> <‪daniel@iogearbox.net‬‏>:‬
>>>>>>
>>>>>> On 10/9/20 6:56 PM, Yaniv Agman wrote:
>>>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
>>>>>>> <‪daniel@iogearbox.net‬‏>:‬
>>>>>>>>
>>>>>>>> [ Cc +Yonghong ]
>>>>>>>>
>>>>>>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
>>>>>>>>> Pulling the latest changes of libbpf and compiling my application with it,
>>>>>>>>> I see the following error:
>>>>>>>>>
>>>>>>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
>>>>>>>>> unknown register name 'r0' in asm
>>>>>>>>>                           : "r0", "r1", "r2", "r3", "r4", "r5");
>>>>>>>>>
>>>>>>>>> The commit which introduced this change is:
>>>>>>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
>>>>>>>>>
>>>>>>>>> I'm not sure if I'm doing something wrong (missing include?), or this
>>>>>>>>> is a genuine error
>>>>>>>>
>>>>>>>> Seems like your clang/llvm version might be too old.
>>>>>>>
>>>>>>> I'm using clang 10.0.1
>>>>>>
>>>>>> Ah, okay, I see. Would this diff do the trick for you?
>>>>>
>>>>> Yes! Now it compiles without any problems!
>>>>
>>>> Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
>>>
>>> Am I the only one confused here?... Yonghong said it should be
>>> supported as early as clang 6, Yaniv is using Clang 10 and is still
>>> getting this error. Let's figure out what's the problem before adding
>>> unnecessary checks.
>>>
>>> I think it's not the clang_major check that helped, rather __bpf__
>>> check. So please hold off on the fix, let's get to the bottom of this
>>> first.
>>
>> I don't see confusion here (maybe other than which minimal clang/llvm version
>> libbpf should support). If we do `#if __clang_major__ >= 6 && defined(__bpf__)`
>> for the final patch, then this means that user passed clang -target bpf and
>> the min supported version for inline assembly was there, otherwise we fall back
>> to bpf_tail_call. In Yaniv's case, he probably had native target with -emit-llvm
>> and then used llc invocation.
> 
> The "-emit-llvm" was the part that we were missing and had to figure
> it out, before we could discuss the fix.

Maybe Yaniv can confirm. I think the following properly happens.
    - clang10 -O2 -g -S -emit-llvm t.c  // This is native compilation 
becasue some header files. Maybe some thing is guarded with x86 specific 
config's which is not available to -target bpf. This is mostly for
tracing programs and Yanic mentions pt_regs which should be related
to tracing.
    - llc -march=bpf t.ll

So guarding the function with __bpf__ should be the one fixing this issue.

guard with clang version >=6 should not hurt and may prevent
compilation failures if people use < 6 llvm with clang -target bpf.
I think most people should already use newer llvm, but who knows.

> 
>>
>>>>>> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
>>>>>> index 2bdb7d6dbad2..31e356831fcf 100644
>>>>>> --- a/tools/lib/bpf/bpf_helpers.h
>>>>>> +++ b/tools/lib/bpf/bpf_helpers.h
>>>>>> @@ -72,6 +72,7 @@
>>>>>>      /*
>>>>>>       * Helper function to perform a tail call with a constant/immediate map slot.
>>>>>>       */
>>>>>> +#if __clang_major__ >= 10 && defined(__bpf__)
>>>>>>      static __always_inline void
>>>>>>      bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>>>>>>      {
>>>>>> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>>>>>>                         :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
>>>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
>>>>>>      }
>>>>>> +#else
>>>>>> +# define bpf_tail_call_static  bpf_tail_call
> 
> bpf_tail_call_static has very specific guarantees, so in cases where
> we can't use inline assembly to satisfy those guarantees, I think we
> should not just silently redefine bpf_tail_call_static as
> bpf_tail_call, rather make compilation fail if someone is attempting
> to use bpf_tail_call_static. _Static_assert could be used to provide a
> better error message here, probably.
> 
>>>>>> +#endif /* __clang_major__ >= 10 && __bpf__ */
>>>>>>
>>>>>>      /*
>>>>>>       * Helper structure used by eBPF C program
>>>>
>>

Re: libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:39 מאת ‪Andrii Nakryiko‬‏
<‪andrii.nakryiko@gmail.com‬‏>:‬
>
> On Fri, Oct 9, 2020 at 11:33 AM Yaniv Agman <yanivagman@gmail.com> wrote:
> >
> > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:21 מאת ‪Daniel Borkmann‬‏
> > <‪daniel@iogearbox.net‬‏>:‬
> > >
> > > On 10/9/20 8:09 PM, Yaniv Agman wrote:
> > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> > > > <‪daniel@iogearbox.net‬‏>:‬
> > > >>
> > > >> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> > > >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> > > >>> <‪daniel@iogearbox.net‬‏>:‬
> > > >>>>
> > > >>>> [ Cc +Yonghong ]
> > > >>>>
> > > >>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> > > >>>>> Pulling the latest changes of libbpf and compiling my application with it,
> > > >>>>> I see the following error:
> > > >>>>>
> > > >>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> > > >>>>> unknown register name 'r0' in asm
> > > >>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
> > > >>>>>
> > > >>>>> The commit which introduced this change is:
> > > >>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> > > >>>>>
> > > >>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> > > >>>>> is a genuine error
> > > >>>>
> > > >>>> Seems like your clang/llvm version might be too old.
> > > >>>
> > > >>> I'm using clang 10.0.1
> > > >>
> > > >> Ah, okay, I see. Would this diff do the trick for you?
> > > >
> > > > Yes! Now it compiles without any problems!
> > >
> > > Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> > >
> >
> > Thanks!
> > Does this happen because I'm first compiling using "emit-llvm" and
> > then using llc?
>
> So this must be the reason, but I'll wait for Yonghong to confirm.
>
> > I wish I could use bpf target directly, but I'm then having problems
> > with includes of asm code (like pt_regs and other stuff)
>
> Are you developing for a 32-bit platform? Or what exactly is the
> problem? I've been trying to solve problems for 32-bit arches recently
> by making libbpf smarter, that relies on CO-RE though. Is CO-RE an
> option for you?
>

Examples for the errors I'm getting:
/lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/atomic.h:177:9:
error: invalid output constraint '+q' in asm
        return xadd(&v->counter, i);
               ^
/lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/cmpxchg.h:234:25:
note: expanded from macro 'xadd'
#define xadd(ptr, inc)          __xadd((ptr), (inc), LOCK_PREFIX)
...

From What I understood, this is a known issue for tracing programs
(like the one I'm developing)
Unfortunately, CO-RE is not (yet) an option.
I'm currently making the move from bcc to libbpf, and our application
needs to support kernel 4.14, and work on all environments.

> [...]

Re: libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:08 מאת ‪Yonghong Song‬‏ <‪yhs@fb.com‬‏>:‬
>
>
>
> On 10/9/20 11:59 AM, Andrii Nakryiko wrote:
> > On Fri, Oct 9, 2020 at 11:41 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >>
> >> On 10/9/20 8:35 PM, Andrii Nakryiko wrote:
> >>> On Fri, Oct 9, 2020 at 11:21 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >>>> On 10/9/20 8:09 PM, Yaniv Agman wrote:
> >>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> >>>>> <‪daniel@iogearbox.net‬‏>:‬
> >>>>>>
> >>>>>> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> >>>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> >>>>>>> <‪daniel@iogearbox.net‬‏>:‬
> >>>>>>>>
> >>>>>>>> [ Cc +Yonghong ]
> >>>>>>>>
> >>>>>>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> >>>>>>>>> Pulling the latest changes of libbpf and compiling my application with it,
> >>>>>>>>> I see the following error:
> >>>>>>>>>
> >>>>>>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> >>>>>>>>> unknown register name 'r0' in asm
> >>>>>>>>>                           : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>>>>>>>
> >>>>>>>>> The commit which introduced this change is:
> >>>>>>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> >>>>>>>>>
> >>>>>>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> >>>>>>>>> is a genuine error
> >>>>>>>>
> >>>>>>>> Seems like your clang/llvm version might be too old.
> >>>>>>>
> >>>>>>> I'm using clang 10.0.1
> >>>>>>
> >>>>>> Ah, okay, I see. Would this diff do the trick for you?
> >>>>>
> >>>>> Yes! Now it compiles without any problems!
> >>>>
> >>>> Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> >>>
> >>> Am I the only one confused here?... Yonghong said it should be
> >>> supported as early as clang 6, Yaniv is using Clang 10 and is still
> >>> getting this error. Let's figure out what's the problem before adding
> >>> unnecessary checks.
> >>>
> >>> I think it's not the clang_major check that helped, rather __bpf__
> >>> check. So please hold off on the fix, let's get to the bottom of this
> >>> first.
> >>
> >> I don't see confusion here (maybe other than which minimal clang/llvm version
> >> libbpf should support). If we do `#if __clang_major__ >= 6 && defined(__bpf__)`
> >> for the final patch, then this means that user passed clang -target bpf and
> >> the min supported version for inline assembly was there, otherwise we fall back
> >> to bpf_tail_call. In Yaniv's case, he probably had native target with -emit-llvm
> >> and then used llc invocation.
> >
> > The "-emit-llvm" was the part that we were missing and had to figure
> > it out, before we could discuss the fix.
>
> Maybe Yaniv can confirm. I think the following properly happens.
>     - clang10 -O2 -g -S -emit-llvm t.c  // This is native compilation
> becasue some header files. Maybe some thing is guarded with x86 specific
> config's which is not available to -target bpf. This is mostly for
> tracing programs and Yanic mentions pt_regs which should be related
> to tracing.
>     - llc -march=bpf t.ll
>

Yes, like I said,  I do use --emit-llvm, and indeed have a tracing program

> So guarding the function with __bpf__ should be the one fixing this issue.
>
> guard with clang version >=6 should not hurt and may prevent
> compilation failures if people use < 6 llvm with clang -target bpf.
> I think most people should already use newer llvm, but who knows.
>
> >
> >>
> >>>>>> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
> >>>>>> index 2bdb7d6dbad2..31e356831fcf 100644
> >>>>>> --- a/tools/lib/bpf/bpf_helpers.h
> >>>>>> +++ b/tools/lib/bpf/bpf_helpers.h
> >>>>>> @@ -72,6 +72,7 @@
> >>>>>>      /*
> >>>>>>       * Helper function to perform a tail call with a constant/immediate map slot.
> >>>>>>       */
> >>>>>> +#if __clang_major__ >= 10 && defined(__bpf__)
> >>>>>>      static __always_inline void
> >>>>>>      bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>>>>>      {
> >>>>>> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>>>>>                         :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
> >>>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>>>>      }
> >>>>>> +#else
> >>>>>> +# define bpf_tail_call_static  bpf_tail_call
> >
> > bpf_tail_call_static has very specific guarantees, so in cases where
> > we can't use inline assembly to satisfy those guarantees, I think we
> > should not just silently redefine bpf_tail_call_static as
> > bpf_tail_call, rather make compilation fail if someone is attempting
> > to use bpf_tail_call_static. _Static_assert could be used to provide a
> > better error message here, probably.
> >
> >>>>>> +#endif /* __clang_major__ >= 10 && __bpf__ */
> >>>>>>
> >>>>>>      /*
> >>>>>>       * Helper structure used by eBPF C program
> >>>>
> >>

Re: libbpf error: unknown register name 'r0' in asm

[Andrii Nakryiko]

On Fri, Oct 9, 2020 at 12:32 PM Yaniv Agman <yanivagman@gmail.com> wrote:
>
> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:39 מאת ‪Andrii Nakryiko‬‏
> <‪andrii.nakryiko@gmail.com‬‏>:‬
> >
> > On Fri, Oct 9, 2020 at 11:33 AM Yaniv Agman <yanivagman@gmail.com> wrote:
> > >
> > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:21 מאת ‪Daniel Borkmann‬‏
> > > <‪daniel@iogearbox.net‬‏>:‬
> > > >
> > > > On 10/9/20 8:09 PM, Yaniv Agman wrote:
> > > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> > > > > <‪daniel@iogearbox.net‬‏>:‬
> > > > >>
> > > > >> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> > > > >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> > > > >>> <‪daniel@iogearbox.net‬‏>:‬
> > > > >>>>
> > > > >>>> [ Cc +Yonghong ]
> > > > >>>>
> > > > >>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> > > > >>>>> Pulling the latest changes of libbpf and compiling my application with it,
> > > > >>>>> I see the following error:
> > > > >>>>>
> > > > >>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> > > > >>>>> unknown register name 'r0' in asm
> > > > >>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
> > > > >>>>>
> > > > >>>>> The commit which introduced this change is:
> > > > >>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> > > > >>>>>
> > > > >>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> > > > >>>>> is a genuine error
> > > > >>>>
> > > > >>>> Seems like your clang/llvm version might be too old.
> > > > >>>
> > > > >>> I'm using clang 10.0.1
> > > > >>
> > > > >> Ah, okay, I see. Would this diff do the trick for you?
> > > > >
> > > > > Yes! Now it compiles without any problems!
> > > >
> > > > Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> > > >
> > >
> > > Thanks!
> > > Does this happen because I'm first compiling using "emit-llvm" and
> > > then using llc?
> >
> > So this must be the reason, but I'll wait for Yonghong to confirm.
> >
> > > I wish I could use bpf target directly, but I'm then having problems
> > > with includes of asm code (like pt_regs and other stuff)
> >
> > Are you developing for a 32-bit platform? Or what exactly is the
> > problem? I've been trying to solve problems for 32-bit arches recently
> > by making libbpf smarter, that relies on CO-RE though. Is CO-RE an
> > option for you?
> >
>
> Examples for the errors I'm getting:
> /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/atomic.h:177:9:
> error: invalid output constraint '+q' in asm
>         return xadd(&v->counter, i);
>                ^
> /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/cmpxchg.h:234:25:
> note: expanded from macro 'xadd'
> #define xadd(ptr, inc)          __xadd((ptr), (inc), LOCK_PREFIX)
> ...
>
> From What I understood, this is a known issue for tracing programs
> (like the one I'm developing)

We do have a bunch of selftests that use pt_regs and include, say,
linux/ptrace.h header. I wonder why we are not seeing these problems.
Selftests, btw, are also built with -emit-llvm and then piping output
to llc.

So.. there must be something else going on. It's hard to guess like
this without seeing the code, but maybe -D__TARGET_ARCH_$(SRCARCH)
during compilation could help, just as an idea.

> Unfortunately, CO-RE is not (yet) an option.
> I'm currently making the move from bcc to libbpf, and our application
> needs to support kernel 4.14, and work on all environments.

Kernel version is not a big problem, it's vmlinux BTF availability
that could be a problem. vmlinux BTF can be added into any version of
kernel with pahole -J, post factum, but that assumes you have some
control over how kernels are built and distributed, of course.

>
> > [...]

Re: libbpf error: unknown register name 'r0' in asm

[Daniel Borkmann]

On 10/9/20 9:33 PM, Yaniv Agman wrote:
> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:08 מאת ‪Yonghong Song‬‏ <‪yhs@fb.com‬‏>:‬
>> On 10/9/20 11:59 AM, Andrii Nakryiko wrote:
>>> On Fri, Oct 9, 2020 at 11:41 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>>>> On 10/9/20 8:35 PM, Andrii Nakryiko wrote:
>>>>> On Fri, Oct 9, 2020 at 11:21 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>>>>>> On 10/9/20 8:09 PM, Yaniv Agman wrote:
>>>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
>>>>>>> <‪daniel@iogearbox.net‬‏>:‬
>>>>>>>>
>>>>>>>> On 10/9/20 6:56 PM, Yaniv Agman wrote:
>>>>>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
>>>>>>>>> <‪daniel@iogearbox.net‬‏>:‬
>>>>>>>>>>
>>>>>>>>>> [ Cc +Yonghong ]
>>>>>>>>>>
>>>>>>>>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
>>>>>>>>>>> Pulling the latest changes of libbpf and compiling my application with it,
>>>>>>>>>>> I see the following error:
>>>>>>>>>>>
>>>>>>>>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
>>>>>>>>>>> unknown register name 'r0' in asm
>>>>>>>>>>>                            : "r0", "r1", "r2", "r3", "r4", "r5");
>>>>>>>>>>>
>>>>>>>>>>> The commit which introduced this change is:
>>>>>>>>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
>>>>>>>>>>>
>>>>>>>>>>> I'm not sure if I'm doing something wrong (missing include?), or this
>>>>>>>>>>> is a genuine error
>>>>>>>>>>
>>>>>>>>>> Seems like your clang/llvm version might be too old.
>>>>>>>>>
>>>>>>>>> I'm using clang 10.0.1
>>>>>>>>
>>>>>>>> Ah, okay, I see. Would this diff do the trick for you?
>>>>>>>
>>>>>>> Yes! Now it compiles without any problems!
>>>>>>
>>>>>> Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
>>>>>
>>>>> Am I the only one confused here?... Yonghong said it should be
>>>>> supported as early as clang 6, Yaniv is using Clang 10 and is still
>>>>> getting this error. Let's figure out what's the problem before adding
>>>>> unnecessary checks.
>>>>>
>>>>> I think it's not the clang_major check that helped, rather __bpf__
>>>>> check. So please hold off on the fix, let's get to the bottom of this
>>>>> first.
>>>>
>>>> I don't see confusion here (maybe other than which minimal clang/llvm version
>>>> libbpf should support). If we do `#if __clang_major__ >= 6 && defined(__bpf__)`
>>>> for the final patch, then this means that user passed clang -target bpf and
>>>> the min supported version for inline assembly was there, otherwise we fall back
>>>> to bpf_tail_call. In Yaniv's case, he probably had native target with -emit-llvm
>>>> and then used llc invocation.
>>>
>>> The "-emit-llvm" was the part that we were missing and had to figure
>>> it out, before we could discuss the fix.
>>
>> Maybe Yaniv can confirm. I think the following properly happens.
>>      - clang10 -O2 -g -S -emit-llvm t.c  // This is native compilation
>> becasue some header files. Maybe some thing is guarded with x86 specific
>> config's which is not available to -target bpf. This is mostly for
>> tracing programs and Yanic mentions pt_regs which should be related
>> to tracing.
>>      - llc -march=bpf t.ll
> 
> Yes, like I said,  I do use --emit-llvm, and indeed have a tracing program
> 
>> So guarding the function with __bpf__ should be the one fixing this issue.
>>
>> guard with clang version >=6 should not hurt and may prevent
>> compilation failures if people use < 6 llvm with clang -target bpf.
>> I think most people should already use newer llvm, but who knows.

Yeah that was my thinking for those stuck for whatever reason on old LLVM.

>>>>>>>> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
>>>>>>>> index 2bdb7d6dbad2..31e356831fcf 100644
>>>>>>>> --- a/tools/lib/bpf/bpf_helpers.h
>>>>>>>> +++ b/tools/lib/bpf/bpf_helpers.h
>>>>>>>> @@ -72,6 +72,7 @@
>>>>>>>>       /*
>>>>>>>>        * Helper function to perform a tail call with a constant/immediate map slot.
>>>>>>>>        */
>>>>>>>> +#if __clang_major__ >= 10 && defined(__bpf__)
>>>>>>>>       static __always_inline void
>>>>>>>>       bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>>>>>>>>       {
>>>>>>>> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>>>>>>>>                          :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
>>>>>>>>                          : "r0", "r1", "r2", "r3", "r4", "r5");
>>>>>>>>       }
>>>>>>>> +#else
>>>>>>>> +# define bpf_tail_call_static  bpf_tail_call
>>>
>>> bpf_tail_call_static has very specific guarantees, so in cases where
>>> we can't use inline assembly to satisfy those guarantees, I think we
>>> should not just silently redefine bpf_tail_call_static as
>>> bpf_tail_call, rather make compilation fail if someone is attempting
>>> to use bpf_tail_call_static. _Static_assert could be used to provide a
>>> better error message here, probably.

Makes sense as well, I was mainly thinking if people include header files in
their project which are shared between tracing & non-tracing, so they compile
just fine, but I can see the point that wrt very specific guarantees, fully
agree. In that sense we should just have it defined with the clang + __bpf__
constraints mentioned earlier.

Thanks,
Daniel

Re: libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:53 מאת ‪Andrii Nakryiko‬‏
<‪andrii.nakryiko@gmail.com‬‏>:‬
>
> On Fri, Oct 9, 2020 at 12:32 PM Yaniv Agman <yanivagman@gmail.com> wrote:
> >
> > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:39 מאת ‪Andrii Nakryiko‬‏
> > <‪andrii.nakryiko@gmail.com‬‏>:‬
> > >
> > > On Fri, Oct 9, 2020 at 11:33 AM Yaniv Agman <yanivagman@gmail.com> wrote:
> > > >
> > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:21 מאת ‪Daniel Borkmann‬‏
> > > > <‪daniel@iogearbox.net‬‏>:‬
> > > > >
> > > > > On 10/9/20 8:09 PM, Yaniv Agman wrote:
> > > > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> > > > > > <‪daniel@iogearbox.net‬‏>:‬
> > > > > >>
> > > > > >> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> > > > > >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> > > > > >>> <‪daniel@iogearbox.net‬‏>:‬
> > > > > >>>>
> > > > > >>>> [ Cc +Yonghong ]
> > > > > >>>>
> > > > > >>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> > > > > >>>>> Pulling the latest changes of libbpf and compiling my application with it,
> > > > > >>>>> I see the following error:
> > > > > >>>>>
> > > > > >>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> > > > > >>>>> unknown register name 'r0' in asm
> > > > > >>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
> > > > > >>>>>
> > > > > >>>>> The commit which introduced this change is:
> > > > > >>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> > > > > >>>>>
> > > > > >>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> > > > > >>>>> is a genuine error
> > > > > >>>>
> > > > > >>>> Seems like your clang/llvm version might be too old.
> > > > > >>>
> > > > > >>> I'm using clang 10.0.1
> > > > > >>
> > > > > >> Ah, okay, I see. Would this diff do the trick for you?
> > > > > >
> > > > > > Yes! Now it compiles without any problems!
> > > > >
> > > > > Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> > > > >
> > > >
> > > > Thanks!
> > > > Does this happen because I'm first compiling using "emit-llvm" and
> > > > then using llc?
> > >
> > > So this must be the reason, but I'll wait for Yonghong to confirm.
> > >
> > > > I wish I could use bpf target directly, but I'm then having problems
> > > > with includes of asm code (like pt_regs and other stuff)
> > >
> > > Are you developing for a 32-bit platform? Or what exactly is the
> > > problem? I've been trying to solve problems for 32-bit arches recently
> > > by making libbpf smarter, that relies on CO-RE though. Is CO-RE an
> > > option for you?
> > >
> >
> > Examples for the errors I'm getting:
> > /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/atomic.h:177:9:
> > error: invalid output constraint '+q' in asm
> >         return xadd(&v->counter, i);
> >                ^
> > /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/cmpxchg.h:234:25:
> > note: expanded from macro 'xadd'
> > #define xadd(ptr, inc)          __xadd((ptr), (inc), LOCK_PREFIX)
> > ...
> >
> > From What I understood, this is a known issue for tracing programs
> > (like the one I'm developing)
>
> We do have a bunch of selftests that use pt_regs and include, say,
> linux/ptrace.h header. I wonder why we are not seeing these problems.
> Selftests, btw, are also built with -emit-llvm and then piping output
> to llc.
>
> So.. there must be something else going on. It's hard to guess like
> this without seeing the code, but maybe -D__TARGET_ARCH_$(SRCARCH)
> during compilation could help, just as an idea.
>

Adding -D__TARGET_ARCH_$(SRCARCH) doesn't seem to help.
If you are interested in having a look at the code,
The code (event_monitor_ebpf.c) including the makefile is here:
https://github.com/yanivagman/tracee/tree/move_to_libbpf/tracee

This is still WIP (the move to libbpf), and libbpf should be added as
a submodule to the project root

> > Unfortunately, CO-RE is not (yet) an option.
> > I'm currently making the move from bcc to libbpf, and our application
> > needs to support kernel 4.14, and work on all environments.
>
> Kernel version is not a big problem, it's vmlinux BTF availability
> that could be a problem. vmlinux BTF can be added into any version of
> kernel with pahole -J, post factum, but that assumes you have some
> control over how kernels are built and distributed, of course.
>
> >
> > > [...]

Re: libbpf error: unknown register name 'r0' in asm

[Andrii Nakryiko]

On Fri, Oct 9, 2020 at 1:24 PM Yaniv Agman <yanivagman@gmail.com> wrote:
>
> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:53 מאת ‪Andrii Nakryiko‬‏
> <‪andrii.nakryiko@gmail.com‬‏>:‬
> >
> > On Fri, Oct 9, 2020 at 12:32 PM Yaniv Agman <yanivagman@gmail.com> wrote:
> > >
> > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:39 מאת ‪Andrii Nakryiko‬‏
> > > <‪andrii.nakryiko@gmail.com‬‏>:‬
> > > >
> > > > On Fri, Oct 9, 2020 at 11:33 AM Yaniv Agman <yanivagman@gmail.com> wrote:
> > > > >
> > > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:21 מאת ‪Daniel Borkmann‬‏
> > > > > <‪daniel@iogearbox.net‬‏>:‬
> > > > > >
> > > > > > On 10/9/20 8:09 PM, Yaniv Agman wrote:
> > > > > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> > > > > > > <‪daniel@iogearbox.net‬‏>:‬
> > > > > > >>
> > > > > > >> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> > > > > > >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> > > > > > >>> <‪daniel@iogearbox.net‬‏>:‬
> > > > > > >>>>
> > > > > > >>>> [ Cc +Yonghong ]
> > > > > > >>>>
> > > > > > >>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> > > > > > >>>>> Pulling the latest changes of libbpf and compiling my application with it,
> > > > > > >>>>> I see the following error:
> > > > > > >>>>>
> > > > > > >>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> > > > > > >>>>> unknown register name 'r0' in asm
> > > > > > >>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
> > > > > > >>>>>
> > > > > > >>>>> The commit which introduced this change is:
> > > > > > >>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> > > > > > >>>>>
> > > > > > >>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> > > > > > >>>>> is a genuine error
> > > > > > >>>>
> > > > > > >>>> Seems like your clang/llvm version might be too old.
> > > > > > >>>
> > > > > > >>> I'm using clang 10.0.1
> > > > > > >>
> > > > > > >> Ah, okay, I see. Would this diff do the trick for you?
> > > > > > >
> > > > > > > Yes! Now it compiles without any problems!
> > > > > >
> > > > > > Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> > > > > >
> > > > >
> > > > > Thanks!
> > > > > Does this happen because I'm first compiling using "emit-llvm" and
> > > > > then using llc?
> > > >
> > > > So this must be the reason, but I'll wait for Yonghong to confirm.
> > > >
> > > > > I wish I could use bpf target directly, but I'm then having problems
> > > > > with includes of asm code (like pt_regs and other stuff)
> > > >
> > > > Are you developing for a 32-bit platform? Or what exactly is the
> > > > problem? I've been trying to solve problems for 32-bit arches recently
> > > > by making libbpf smarter, that relies on CO-RE though. Is CO-RE an
> > > > option for you?
> > > >
> > >
> > > Examples for the errors I'm getting:
> > > /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/atomic.h:177:9:
> > > error: invalid output constraint '+q' in asm
> > >         return xadd(&v->counter, i);
> > >                ^
> > > /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/cmpxchg.h:234:25:
> > > note: expanded from macro 'xadd'
> > > #define xadd(ptr, inc)          __xadd((ptr), (inc), LOCK_PREFIX)
> > > ...
> > >
> > > From What I understood, this is a known issue for tracing programs
> > > (like the one I'm developing)
> >
> > We do have a bunch of selftests that use pt_regs and include, say,
> > linux/ptrace.h header. I wonder why we are not seeing these problems.
> > Selftests, btw, are also built with -emit-llvm and then piping output
> > to llc.
> >
> > So.. there must be something else going on. It's hard to guess like
> > this without seeing the code, but maybe -D__TARGET_ARCH_$(SRCARCH)
> > during compilation could help, just as an idea.
> >
>
> Adding -D__TARGET_ARCH_$(SRCARCH) doesn't seem to help.
> If you are interested in having a look at the code,
> The code (event_monitor_ebpf.c) including the makefile is here:
> https://github.com/yanivagman/tracee/tree/move_to_libbpf/tracee

You are including a lot of kernel headers in that code. If any of them
pulls in some x86-specific stuff, you would get this problem. Try to
minimize the amount of includes you have there, maybe the issue will
go away.

If not, one other way would be to generate vmlinux.h for your specific
kernel build. It will have memory layout of all the structs identical
to what you get from kernel headers. Then you can replace all those
includes with a single #include "vmlinux.h". You can disable CO-RE
part with #define BPF_NO_PRESERVE_ACCESS_INDEX before vmlinux.h is
included.

But I'm a bit puzzled how your approach is going to work without CO-RE
if you ever intend to run your program on more than one build of the
kernel. Different kernel builds/versions might have different memory
layouts of the structs you are relying on, so you'd need to compile
them for each kernel build separately. So it's either CO-RE or
on-the-host compilation (what BCC is doing). And if you are moving
away from BCC, but not adopting CO-RE, I'm not sure how you are going
to deal with that issue.


>
> This is still WIP (the move to libbpf), and libbpf should be added as
> a submodule to the project root
>
> > > Unfortunately, CO-RE is not (yet) an option.
> > > I'm currently making the move from bcc to libbpf, and our application
> > > needs to support kernel 4.14, and work on all environments.
> >
> > Kernel version is not a big problem, it's vmlinux BTF availability
> > that could be a problem. vmlinux BTF can be added into any version of
> > kernel with pahole -J, post factum, but that assumes you have some
> > control over how kernels are built and distributed, of course.
> >
> > >
> > > > [...]

Re: libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

‫בתאריך יום ב׳, 12 באוק׳ 2020 ב-23:03 מאת ‪Andrii Nakryiko‬‏
<‪andrii.nakryiko@gmail.com‬‏>:‬
>
> On Fri, Oct 9, 2020 at 1:24 PM Yaniv Agman <yanivagman@gmail.com> wrote:
> >
> > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:53 מאת ‪Andrii Nakryiko‬‏
> > <‪andrii.nakryiko@gmail.com‬‏>:‬
> > >
> > > On Fri, Oct 9, 2020 at 12:32 PM Yaniv Agman <yanivagman@gmail.com> wrote:
> > > >
> > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:39 מאת ‪Andrii Nakryiko‬‏
> > > > <‪andrii.nakryiko@gmail.com‬‏>:‬
> > > > >
> > > > > On Fri, Oct 9, 2020 at 11:33 AM Yaniv Agman <yanivagman@gmail.com> wrote:
> > > > > >
> > > > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:21 מאת ‪Daniel Borkmann‬‏
> > > > > > <‪daniel@iogearbox.net‬‏>:‬
> > > > > > >
> > > > > > > On 10/9/20 8:09 PM, Yaniv Agman wrote:
> > > > > > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> > > > > > > > <‪daniel@iogearbox.net‬‏>:‬
> > > > > > > >>
> > > > > > > >> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> > > > > > > >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> > > > > > > >>> <‪daniel@iogearbox.net‬‏>:‬
> > > > > > > >>>>
> > > > > > > >>>> [ Cc +Yonghong ]
> > > > > > > >>>>
> > > > > > > >>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> > > > > > > >>>>> Pulling the latest changes of libbpf and compiling my application with it,
> > > > > > > >>>>> I see the following error:
> > > > > > > >>>>>
> > > > > > > >>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> > > > > > > >>>>> unknown register name 'r0' in asm
> > > > > > > >>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
> > > > > > > >>>>>
> > > > > > > >>>>> The commit which introduced this change is:
> > > > > > > >>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> > > > > > > >>>>>
> > > > > > > >>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> > > > > > > >>>>> is a genuine error
> > > > > > > >>>>
> > > > > > > >>>> Seems like your clang/llvm version might be too old.
> > > > > > > >>>
> > > > > > > >>> I'm using clang 10.0.1
> > > > > > > >>
> > > > > > > >> Ah, okay, I see. Would this diff do the trick for you?
> > > > > > > >
> > > > > > > > Yes! Now it compiles without any problems!
> > > > > > >
> > > > > > > Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> > > > > > >
> > > > > >
> > > > > > Thanks!
> > > > > > Does this happen because I'm first compiling using "emit-llvm" and
> > > > > > then using llc?
> > > > >
> > > > > So this must be the reason, but I'll wait for Yonghong to confirm.
> > > > >
> > > > > > I wish I could use bpf target directly, but I'm then having problems
> > > > > > with includes of asm code (like pt_regs and other stuff)
> > > > >
> > > > > Are you developing for a 32-bit platform? Or what exactly is the
> > > > > problem? I've been trying to solve problems for 32-bit arches recently
> > > > > by making libbpf smarter, that relies on CO-RE though. Is CO-RE an
> > > > > option for you?
> > > > >
> > > >
> > > > Examples for the errors I'm getting:
> > > > /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/atomic.h:177:9:
> > > > error: invalid output constraint '+q' in asm
> > > >         return xadd(&v->counter, i);
> > > >                ^
> > > > /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/cmpxchg.h:234:25:
> > > > note: expanded from macro 'xadd'
> > > > #define xadd(ptr, inc)          __xadd((ptr), (inc), LOCK_PREFIX)
> > > > ...
> > > >
> > > > From What I understood, this is a known issue for tracing programs
> > > > (like the one I'm developing)
> > >
> > > We do have a bunch of selftests that use pt_regs and include, say,
> > > linux/ptrace.h header. I wonder why we are not seeing these problems.
> > > Selftests, btw, are also built with -emit-llvm and then piping output
> > > to llc.
> > >
> > > So.. there must be something else going on. It's hard to guess like
> > > this without seeing the code, but maybe -D__TARGET_ARCH_$(SRCARCH)
> > > during compilation could help, just as an idea.
> > >
> >
> > Adding -D__TARGET_ARCH_$(SRCARCH) doesn't seem to help.
> > If you are interested in having a look at the code,
> > The code (event_monitor_ebpf.c) including the makefile is here:
> > https://github.com/yanivagman/tracee/tree/move_to_libbpf/tracee
>
> You are including a lot of kernel headers in that code. If any of them
> pulls in some x86-specific stuff, you would get this problem. Try to
> minimize the amount of includes you have there, maybe the issue will
> go away.
>

I only added an include when I needed it, so I'm not sure I can remove
any of them...

> If not, one other way would be to generate vmlinux.h for your specific
> kernel build. It will have memory layout of all the structs identical
> to what you get from kernel headers. Then you can replace all those
> includes with a single #include "vmlinux.h". You can disable CO-RE
> part with #define BPF_NO_PRESERVE_ACCESS_INDEX before vmlinux.h is
> included.
>

My application is supposed to run on every linux environment with kernel > 4.14.
The only requirement from the user is having clang and kernel headers.
Is there a way to generate vmlinux.h from the given kernel headers?
From my understanding, vmlinux.h can only be generated from vmlinux
using bpftool,
but I don't want to require the users to supply vmlinux as well.

> But I'm a bit puzzled how your approach is going to work without CO-RE
> if you ever intend to run your program on more than one build of the
> kernel. Different kernel builds/versions might have different memory
> layouts of the structs you are relying on, so you'd need to compile
> them for each kernel build separately. So it's either CO-RE or
> on-the-host compilation (what BCC is doing). And if you are moving
> away from BCC, but not adopting CO-RE, I'm not sure how you are going
> to deal with that issue.
>

The idea is to compile the code once on the target node, then use the
created object file in subsequent runs.
Unlike bcc, which compiles the bpf code on every run,
with this approach, we "install" the application once, then the
program should run immediately without any further required
compilation
The end goal is to support CO-RE, after most distros will support it
out of the box, and the user won't have to supply vmlinux for the
program to work

>
> >
> > This is still WIP (the move to libbpf), and libbpf should be added as
> > a submodule to the project root
> >
> > > > Unfortunately, CO-RE is not (yet) an option.
> > > > I'm currently making the move from bcc to libbpf, and our application
> > > > needs to support kernel 4.14, and work on all environments.
> > >
> > > Kernel version is not a big problem, it's vmlinux BTF availability
> > > that could be a problem. vmlinux BTF can be added into any version of
> > > kernel with pahole -J, post factum, but that assumes you have some
> > > control over how kernels are built and distributed, of course.
> > >
> > > >
> > > > > [...]

Re: libbpf error: unknown register name 'r0' in asm

[Andrii Nakryiko]

On Mon, Oct 12, 2020 at 2:49 PM Yaniv Agman <yanivagman@gmail.com> wrote:
>
> ‫בתאריך יום ב׳, 12 באוק׳ 2020 ב-23:03 מאת ‪Andrii Nakryiko‬‏
> <‪andrii.nakryiko@gmail.com‬‏>:‬
> >
> > On Fri, Oct 9, 2020 at 1:24 PM Yaniv Agman <yanivagman@gmail.com> wrote:
> > >
> > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:53 מאת ‪Andrii Nakryiko‬‏
> > > <‪andrii.nakryiko@gmail.com‬‏>:‬
> > > >
> > > > On Fri, Oct 9, 2020 at 12:32 PM Yaniv Agman <yanivagman@gmail.com> wrote:
> > > > >
> > > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:39 מאת ‪Andrii Nakryiko‬‏
> > > > > <‪andrii.nakryiko@gmail.com‬‏>:‬
> > > > > >
> > > > > > On Fri, Oct 9, 2020 at 11:33 AM Yaniv Agman <yanivagman@gmail.com> wrote:
> > > > > > >
> > > > > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:21 מאת ‪Daniel Borkmann‬‏
> > > > > > > <‪daniel@iogearbox.net‬‏>:‬
> > > > > > > >
> > > > > > > > On 10/9/20 8:09 PM, Yaniv Agman wrote:
> > > > > > > > > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> > > > > > > > > <‪daniel@iogearbox.net‬‏>:‬
> > > > > > > > >>
> > > > > > > > >> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> > > > > > > > >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> > > > > > > > >>> <‪daniel@iogearbox.net‬‏>:‬
> > > > > > > > >>>>
> > > > > > > > >>>> [ Cc +Yonghong ]
> > > > > > > > >>>>
> > > > > > > > >>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> > > > > > > > >>>>> Pulling the latest changes of libbpf and compiling my application with it,
> > > > > > > > >>>>> I see the following error:
> > > > > > > > >>>>>
> > > > > > > > >>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> > > > > > > > >>>>> unknown register name 'r0' in asm
> > > > > > > > >>>>>                         : "r0", "r1", "r2", "r3", "r4", "r5");
> > > > > > > > >>>>>
> > > > > > > > >>>>> The commit which introduced this change is:
> > > > > > > > >>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> > > > > > > > >>>>>
> > > > > > > > >>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> > > > > > > > >>>>> is a genuine error
> > > > > > > > >>>>
> > > > > > > > >>>> Seems like your clang/llvm version might be too old.
> > > > > > > > >>>
> > > > > > > > >>> I'm using clang 10.0.1
> > > > > > > > >>
> > > > > > > > >> Ah, okay, I see. Would this diff do the trick for you?
> > > > > > > > >
> > > > > > > > > Yes! Now it compiles without any problems!
> > > > > > > >
> > > > > > > > Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> > > > > > > >
> > > > > > >
> > > > > > > Thanks!
> > > > > > > Does this happen because I'm first compiling using "emit-llvm" and
> > > > > > > then using llc?
> > > > > >
> > > > > > So this must be the reason, but I'll wait for Yonghong to confirm.
> > > > > >
> > > > > > > I wish I could use bpf target directly, but I'm then having problems
> > > > > > > with includes of asm code (like pt_regs and other stuff)
> > > > > >
> > > > > > Are you developing for a 32-bit platform? Or what exactly is the
> > > > > > problem? I've been trying to solve problems for 32-bit arches recently
> > > > > > by making libbpf smarter, that relies on CO-RE though. Is CO-RE an
> > > > > > option for you?
> > > > > >
> > > > >
> > > > > Examples for the errors I'm getting:
> > > > > /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/atomic.h:177:9:
> > > > > error: invalid output constraint '+q' in asm
> > > > >         return xadd(&v->counter, i);
> > > > >                ^
> > > > > /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/cmpxchg.h:234:25:
> > > > > note: expanded from macro 'xadd'
> > > > > #define xadd(ptr, inc)          __xadd((ptr), (inc), LOCK_PREFIX)
> > > > > ...
> > > > >
> > > > > From What I understood, this is a known issue for tracing programs
> > > > > (like the one I'm developing)
> > > >
> > > > We do have a bunch of selftests that use pt_regs and include, say,
> > > > linux/ptrace.h header. I wonder why we are not seeing these problems.
> > > > Selftests, btw, are also built with -emit-llvm and then piping output
> > > > to llc.
> > > >
> > > > So.. there must be something else going on. It's hard to guess like
> > > > this without seeing the code, but maybe -D__TARGET_ARCH_$(SRCARCH)
> > > > during compilation could help, just as an idea.
> > > >
> > >
> > > Adding -D__TARGET_ARCH_$(SRCARCH) doesn't seem to help.
> > > If you are interested in having a look at the code,
> > > The code (event_monitor_ebpf.c) including the makefile is here:
> > > https://github.com/yanivagman/tracee/tree/move_to_libbpf/tracee
> >
> > You are including a lot of kernel headers in that code. If any of them
> > pulls in some x86-specific stuff, you would get this problem. Try to
> > minimize the amount of includes you have there, maybe the issue will
> > go away.
> >
>
> I only added an include when I needed it, so I'm not sure I can remove
> any of them...
>
> > If not, one other way would be to generate vmlinux.h for your specific
> > kernel build. It will have memory layout of all the structs identical
> > to what you get from kernel headers. Then you can replace all those
> > includes with a single #include "vmlinux.h". You can disable CO-RE
> > part with #define BPF_NO_PRESERVE_ACCESS_INDEX before vmlinux.h is
> > included.
> >
>
> My application is supposed to run on every linux environment with kernel > 4.14.
> The only requirement from the user is having clang and kernel headers.
> Is there a way to generate vmlinux.h from the given kernel headers?

nope

> From my understanding, vmlinux.h can only be generated from vmlinux
> using bpftool,
> but I don't want to require the users to supply vmlinux as well.

you need either a vmlinux image or just .BTF binary section contents,
which you can extract with objdump. But either way, binary BTF data is
needed, bpftool won't parse/compile C headers to generate vmlinux.h.

>
> > But I'm a bit puzzled how your approach is going to work without CO-RE
> > if you ever intend to run your program on more than one build of the
> > kernel. Different kernel builds/versions might have different memory
> > layouts of the structs you are relying on, so you'd need to compile
> > them for each kernel build separately. So it's either CO-RE or
> > on-the-host compilation (what BCC is doing). And if you are moving
> > away from BCC, but not adopting CO-RE, I'm not sure how you are going
> > to deal with that issue.
> >
>
> The idea is to compile the code once on the target node, then use the
> created object file in subsequent runs.
> Unlike bcc, which compiles the bpf code on every run,
> with this approach, we "install" the application once, then the
> program should run immediately without any further required
> compilation

I see, so there is on-the-box compilation involved. If you can make
sure you have vmlinux with DWARF on each target box, you can generate
BTF on your own with pahole, I suppose. But vmlinux with DWARF becomes
another requirement that you might not be able to satisfy.

> The end goal is to support CO-RE, after most distros will support it
> out of the box, and the user won't have to supply vmlinux for the
> program to work
>
> >
> > >
> > > This is still WIP (the move to libbpf), and libbpf should be added as
> > > a submodule to the project root
> > >
> > > > > Unfortunately, CO-RE is not (yet) an option.
> > > > > I'm currently making the move from bcc to libbpf, and our application
> > > > > needs to support kernel 4.14, and work on all environments.
> > > >
> > > > Kernel version is not a big problem, it's vmlinux BTF availability
> > > > that could be a problem. vmlinux BTF can be added into any version of
> > > > kernel with pahole -J, post factum, but that assumes you have some
> > > > control over how kernels are built and distributed, of course.
> > > >
> > > > >
> > > > > > [...]

Re: libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:58 מאת ‪Daniel Borkmann‬‏
<‪daniel@iogearbox.net‬‏>:‬
>
> On 10/9/20 9:33 PM, Yaniv Agman wrote:
> > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:08 מאת ‪Yonghong Song‬‏ <‪yhs@fb.com‬‏>:‬
> >> On 10/9/20 11:59 AM, Andrii Nakryiko wrote:
> >>> On Fri, Oct 9, 2020 at 11:41 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >>>> On 10/9/20 8:35 PM, Andrii Nakryiko wrote:
> >>>>> On Fri, Oct 9, 2020 at 11:21 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >>>>>> On 10/9/20 8:09 PM, Yaniv Agman wrote:
> >>>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> >>>>>>> <‪daniel@iogearbox.net‬‏>:‬
> >>>>>>>>
> >>>>>>>> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> >>>>>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> >>>>>>>>> <‪daniel@iogearbox.net‬‏>:‬
> >>>>>>>>>>
> >>>>>>>>>> [ Cc +Yonghong ]
> >>>>>>>>>>
> >>>>>>>>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> >>>>>>>>>>> Pulling the latest changes of libbpf and compiling my application with it,
> >>>>>>>>>>> I see the following error:
> >>>>>>>>>>>
> >>>>>>>>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> >>>>>>>>>>> unknown register name 'r0' in asm
> >>>>>>>>>>>                            : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>>>>>>>>>
> >>>>>>>>>>> The commit which introduced this change is:
> >>>>>>>>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> >>>>>>>>>>>
> >>>>>>>>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> >>>>>>>>>>> is a genuine error
> >>>>>>>>>>
> >>>>>>>>>> Seems like your clang/llvm version might be too old.
> >>>>>>>>>
> >>>>>>>>> I'm using clang 10.0.1
> >>>>>>>>
> >>>>>>>> Ah, okay, I see. Would this diff do the trick for you?
> >>>>>>>
> >>>>>>> Yes! Now it compiles without any problems!
> >>>>>>
> >>>>>> Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> >>>>>
> >>>>> Am I the only one confused here?... Yonghong said it should be
> >>>>> supported as early as clang 6, Yaniv is using Clang 10 and is still
> >>>>> getting this error. Let's figure out what's the problem before adding
> >>>>> unnecessary checks.
> >>>>>
> >>>>> I think it's not the clang_major check that helped, rather __bpf__
> >>>>> check. So please hold off on the fix, let's get to the bottom of this
> >>>>> first.
> >>>>
> >>>> I don't see confusion here (maybe other than which minimal clang/llvm version
> >>>> libbpf should support). If we do `#if __clang_major__ >= 6 && defined(__bpf__)`
> >>>> for the final patch, then this means that user passed clang -target bpf and
> >>>> the min supported version for inline assembly was there, otherwise we fall back
> >>>> to bpf_tail_call. In Yaniv's case, he probably had native target with -emit-llvm
> >>>> and then used llc invocation.
> >>>
> >>> The "-emit-llvm" was the part that we were missing and had to figure
> >>> it out, before we could discuss the fix.
> >>
> >> Maybe Yaniv can confirm. I think the following properly happens.
> >>      - clang10 -O2 -g -S -emit-llvm t.c  // This is native compilation
> >> becasue some header files. Maybe some thing is guarded with x86 specific
> >> config's which is not available to -target bpf. This is mostly for
> >> tracing programs and Yanic mentions pt_regs which should be related
> >> to tracing.
> >>      - llc -march=bpf t.ll
> >
> > Yes, like I said,  I do use --emit-llvm, and indeed have a tracing program
> >
> >> So guarding the function with __bpf__ should be the one fixing this issue.
> >>
> >> guard with clang version >=6 should not hurt and may prevent
> >> compilation failures if people use < 6 llvm with clang -target bpf.
> >> I think most people should already use newer llvm, but who knows.
>
> Yeah that was my thinking for those stuck for whatever reason on old LLVM.
>
> >>>>>>>> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
> >>>>>>>> index 2bdb7d6dbad2..31e356831fcf 100644
> >>>>>>>> --- a/tools/lib/bpf/bpf_helpers.h
> >>>>>>>> +++ b/tools/lib/bpf/bpf_helpers.h
> >>>>>>>> @@ -72,6 +72,7 @@
> >>>>>>>>       /*
> >>>>>>>>        * Helper function to perform a tail call with a constant/immediate map slot.
> >>>>>>>>        */
> >>>>>>>> +#if __clang_major__ >= 10 && defined(__bpf__)
> >>>>>>>>       static __always_inline void
> >>>>>>>>       bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>>>>>>>       {
> >>>>>>>> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>>>>>>>                          :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
> >>>>>>>>                          : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>>>>>>       }
> >>>>>>>> +#else
> >>>>>>>> +# define bpf_tail_call_static  bpf_tail_call
> >>>
> >>> bpf_tail_call_static has very specific guarantees, so in cases where
> >>> we can't use inline assembly to satisfy those guarantees, I think we
> >>> should not just silently redefine bpf_tail_call_static as
> >>> bpf_tail_call, rather make compilation fail if someone is attempting
> >>> to use bpf_tail_call_static. _Static_assert could be used to provide a
> >>> better error message here, probably.
>
> Makes sense as well, I was mainly thinking if people include header files in
> their project which are shared between tracing & non-tracing, so they compile
> just fine, but I can see the point that wrt very specific guarantees, fully
> agree. In that sense we should just have it defined with the clang + __bpf__
> constraints mentioned earlier.
>
> Thanks,
> Daniel

Hi Daniel,

Is this change going to happen?
I'm still having a compilation error when using master branch

Thanks,
Yaniv

Re: libbpf error: unknown register name 'r0' in asm

[Daniel Borkmann]

On 10/21/20 11:43 AM, Yaniv Agman wrote:
> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:58 מאת ‪Daniel Borkmann‬‏
> <‪daniel@iogearbox.net‬‏>:‬
>> On 10/9/20 9:33 PM, Yaniv Agman wrote:
>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:08 מאת ‪Yonghong Song‬‏ <‪yhs@fb.com‬‏>:‬
>>>> On 10/9/20 11:59 AM, Andrii Nakryiko wrote:
>>>>> On Fri, Oct 9, 2020 at 11:41 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>>>>>> On 10/9/20 8:35 PM, Andrii Nakryiko wrote:
>>>>>>> On Fri, Oct 9, 2020 at 11:21 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>>>>>>>> On 10/9/20 8:09 PM, Yaniv Agman wrote:
>>>>>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
>>>>>>>>> <‪daniel@iogearbox.net‬‏>:‬
>>>>>>>>>>
>>>>>>>>>> On 10/9/20 6:56 PM, Yaniv Agman wrote:
>>>>>>>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
>>>>>>>>>>> <‪daniel@iogearbox.net‬‏>:‬
>>>>>>>>>>>>
>>>>>>>>>>>> [ Cc +Yonghong ]
>>>>>>>>>>>>
>>>>>>>>>>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
>>>>>>>>>>>>> Pulling the latest changes of libbpf and compiling my application with it,
>>>>>>>>>>>>> I see the following error:
>>>>>>>>>>>>>
>>>>>>>>>>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
>>>>>>>>>>>>> unknown register name 'r0' in asm
>>>>>>>>>>>>>                             : "r0", "r1", "r2", "r3", "r4", "r5");
>>>>>>>>>>>>>
>>>>>>>>>>>>> The commit which introduced this change is:
>>>>>>>>>>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm not sure if I'm doing something wrong (missing include?), or this
>>>>>>>>>>>>> is a genuine error
>>>>>>>>>>>>
>>>>>>>>>>>> Seems like your clang/llvm version might be too old.
>>>>>>>>>>>
>>>>>>>>>>> I'm using clang 10.0.1
>>>>>>>>>>
>>>>>>>>>> Ah, okay, I see. Would this diff do the trick for you?
>>>>>>>>>
>>>>>>>>> Yes! Now it compiles without any problems!
>>>>>>>>
>>>>>>>> Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
>>>>>>>
>>>>>>> Am I the only one confused here?... Yonghong said it should be
>>>>>>> supported as early as clang 6, Yaniv is using Clang 10 and is still
>>>>>>> getting this error. Let's figure out what's the problem before adding
>>>>>>> unnecessary checks.
>>>>>>>
>>>>>>> I think it's not the clang_major check that helped, rather __bpf__
>>>>>>> check. So please hold off on the fix, let's get to the bottom of this
>>>>>>> first.
>>>>>>
>>>>>> I don't see confusion here (maybe other than which minimal clang/llvm version
>>>>>> libbpf should support). If we do `#if __clang_major__ >= 6 && defined(__bpf__)`
>>>>>> for the final patch, then this means that user passed clang -target bpf and
>>>>>> the min supported version for inline assembly was there, otherwise we fall back
>>>>>> to bpf_tail_call. In Yaniv's case, he probably had native target with -emit-llvm
>>>>>> and then used llc invocation.
>>>>>
>>>>> The "-emit-llvm" was the part that we were missing and had to figure
>>>>> it out, before we could discuss the fix.
>>>>
>>>> Maybe Yaniv can confirm. I think the following properly happens.
>>>>       - clang10 -O2 -g -S -emit-llvm t.c  // This is native compilation
>>>> becasue some header files. Maybe some thing is guarded with x86 specific
>>>> config's which is not available to -target bpf. This is mostly for
>>>> tracing programs and Yanic mentions pt_regs which should be related
>>>> to tracing.
>>>>       - llc -march=bpf t.ll
>>>
>>> Yes, like I said,  I do use --emit-llvm, and indeed have a tracing program
>>>
>>>> So guarding the function with __bpf__ should be the one fixing this issue.
>>>>
>>>> guard with clang version >=6 should not hurt and may prevent
>>>> compilation failures if people use < 6 llvm with clang -target bpf.
>>>> I think most people should already use newer llvm, but who knows.
>>
>> Yeah that was my thinking for those stuck for whatever reason on old LLVM.
>>
>>>>>>>>>> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
>>>>>>>>>> index 2bdb7d6dbad2..31e356831fcf 100644
>>>>>>>>>> --- a/tools/lib/bpf/bpf_helpers.h
>>>>>>>>>> +++ b/tools/lib/bpf/bpf_helpers.h
>>>>>>>>>> @@ -72,6 +72,7 @@
>>>>>>>>>>        /*
>>>>>>>>>>         * Helper function to perform a tail call with a constant/immediate map slot.
>>>>>>>>>>         */
>>>>>>>>>> +#if __clang_major__ >= 10 && defined(__bpf__)
>>>>>>>>>>        static __always_inline void
>>>>>>>>>>        bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>>>>>>>>>>        {
>>>>>>>>>> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
>>>>>>>>>>                           :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
>>>>>>>>>>                           : "r0", "r1", "r2", "r3", "r4", "r5");
>>>>>>>>>>        }
>>>>>>>>>> +#else
>>>>>>>>>> +# define bpf_tail_call_static  bpf_tail_call
>>>>>
>>>>> bpf_tail_call_static has very specific guarantees, so in cases where
>>>>> we can't use inline assembly to satisfy those guarantees, I think we
>>>>> should not just silently redefine bpf_tail_call_static as
>>>>> bpf_tail_call, rather make compilation fail if someone is attempting
>>>>> to use bpf_tail_call_static. _Static_assert could be used to provide a
>>>>> better error message here, probably.
>>
>> Makes sense as well, I was mainly thinking if people include header files in
>> their project which are shared between tracing & non-tracing, so they compile
>> just fine, but I can see the point that wrt very specific guarantees, fully
>> agree. In that sense we should just have it defined with the clang + __bpf__
>> constraints mentioned earlier.
> 
> Is this change going to happen?
> I'm still having a compilation error when using master branch

Yeah, I'll submit something tonight.

Thanks,
Daniel

Re: libbpf error: unknown register name 'r0' in asm

[Yaniv Agman]

‫בתאריך יום ד׳, 21 באוק׳ 2020 ב-20:18 מאת ‪Daniel Borkmann‬‏
<‪daniel@iogearbox.net‬‏>:‬
>
> On 10/21/20 11:43 AM, Yaniv Agman wrote:
> > ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:58 מאת ‪Daniel Borkmann‬‏
> > <‪daniel@iogearbox.net‬‏>:‬
> >> On 10/9/20 9:33 PM, Yaniv Agman wrote:
> >>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:08 מאת ‪Yonghong Song‬‏ <‪yhs@fb.com‬‏>:‬
> >>>> On 10/9/20 11:59 AM, Andrii Nakryiko wrote:
> >>>>> On Fri, Oct 9, 2020 at 11:41 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >>>>>> On 10/9/20 8:35 PM, Andrii Nakryiko wrote:
> >>>>>>> On Fri, Oct 9, 2020 at 11:21 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >>>>>>>> On 10/9/20 8:09 PM, Yaniv Agman wrote:
> >>>>>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת ‪Daniel Borkmann‬‏
> >>>>>>>>> <‪daniel@iogearbox.net‬‏>:‬
> >>>>>>>>>>
> >>>>>>>>>> On 10/9/20 6:56 PM, Yaniv Agman wrote:
> >>>>>>>>>>> ‫בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת ‪Daniel Borkmann‬‏
> >>>>>>>>>>> <‪daniel@iogearbox.net‬‏>:‬
> >>>>>>>>>>>>
> >>>>>>>>>>>> [ Cc +Yonghong ]
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 10/9/20 6:05 PM, Yaniv Agman wrote:
> >>>>>>>>>>>>> Pulling the latest changes of libbpf and compiling my application with it,
> >>>>>>>>>>>>> I see the following error:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error:
> >>>>>>>>>>>>> unknown register name 'r0' in asm
> >>>>>>>>>>>>>                             : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The commit which introduced this change is:
> >>>>>>>>>>>>> 80c7838600d39891f274e2f7508b95a75e4227c1
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I'm not sure if I'm doing something wrong (missing include?), or this
> >>>>>>>>>>>>> is a genuine error
> >>>>>>>>>>>>
> >>>>>>>>>>>> Seems like your clang/llvm version might be too old.
> >>>>>>>>>>>
> >>>>>>>>>>> I'm using clang 10.0.1
> >>>>>>>>>>
> >>>>>>>>>> Ah, okay, I see. Would this diff do the trick for you?
> >>>>>>>>>
> >>>>>>>>> Yes! Now it compiles without any problems!
> >>>>>>>>
> >>>>>>>> Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned.
> >>>>>>>
> >>>>>>> Am I the only one confused here?... Yonghong said it should be
> >>>>>>> supported as early as clang 6, Yaniv is using Clang 10 and is still
> >>>>>>> getting this error. Let's figure out what's the problem before adding
> >>>>>>> unnecessary checks.
> >>>>>>>
> >>>>>>> I think it's not the clang_major check that helped, rather __bpf__
> >>>>>>> check. So please hold off on the fix, let's get to the bottom of this
> >>>>>>> first.
> >>>>>>
> >>>>>> I don't see confusion here (maybe other than which minimal clang/llvm version
> >>>>>> libbpf should support). If we do `#if __clang_major__ >= 6 && defined(__bpf__)`
> >>>>>> for the final patch, then this means that user passed clang -target bpf and
> >>>>>> the min supported version for inline assembly was there, otherwise we fall back
> >>>>>> to bpf_tail_call. In Yaniv's case, he probably had native target with -emit-llvm
> >>>>>> and then used llc invocation.
> >>>>>
> >>>>> The "-emit-llvm" was the part that we were missing and had to figure
> >>>>> it out, before we could discuss the fix.
> >>>>
> >>>> Maybe Yaniv can confirm. I think the following properly happens.
> >>>>       - clang10 -O2 -g -S -emit-llvm t.c  // This is native compilation
> >>>> becasue some header files. Maybe some thing is guarded with x86 specific
> >>>> config's which is not available to -target bpf. This is mostly for
> >>>> tracing programs and Yanic mentions pt_regs which should be related
> >>>> to tracing.
> >>>>       - llc -march=bpf t.ll
> >>>
> >>> Yes, like I said,  I do use --emit-llvm, and indeed have a tracing program
> >>>
> >>>> So guarding the function with __bpf__ should be the one fixing this issue.
> >>>>
> >>>> guard with clang version >=6 should not hurt and may prevent
> >>>> compilation failures if people use < 6 llvm with clang -target bpf.
> >>>> I think most people should already use newer llvm, but who knows.
> >>
> >> Yeah that was my thinking for those stuck for whatever reason on old LLVM.
> >>
> >>>>>>>>>> diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
> >>>>>>>>>> index 2bdb7d6dbad2..31e356831fcf 100644
> >>>>>>>>>> --- a/tools/lib/bpf/bpf_helpers.h
> >>>>>>>>>> +++ b/tools/lib/bpf/bpf_helpers.h
> >>>>>>>>>> @@ -72,6 +72,7 @@
> >>>>>>>>>>        /*
> >>>>>>>>>>         * Helper function to perform a tail call with a constant/immediate map slot.
> >>>>>>>>>>         */
> >>>>>>>>>> +#if __clang_major__ >= 10 && defined(__bpf__)
> >>>>>>>>>>        static __always_inline void
> >>>>>>>>>>        bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>>>>>>>>>        {
> >>>>>>>>>> @@ -98,6 +99,9 @@ bpf_tail_call_static(void *ctx, const void *map, const __u32 slot)
> >>>>>>>>>>                           :: [ctx]"r"(ctx), [map]"r"(map), [slot]"i"(slot)
> >>>>>>>>>>                           : "r0", "r1", "r2", "r3", "r4", "r5");
> >>>>>>>>>>        }
> >>>>>>>>>> +#else
> >>>>>>>>>> +# define bpf_tail_call_static  bpf_tail_call
> >>>>>
> >>>>> bpf_tail_call_static has very specific guarantees, so in cases where
> >>>>> we can't use inline assembly to satisfy those guarantees, I think we
> >>>>> should not just silently redefine bpf_tail_call_static as
> >>>>> bpf_tail_call, rather make compilation fail if someone is attempting
> >>>>> to use bpf_tail_call_static. _Static_assert could be used to provide a
> >>>>> better error message here, probably.
> >>
> >> Makes sense as well, I was mainly thinking if people include header files in
> >> their project which are shared between tracing & non-tracing, so they compile
> >> just fine, but I can see the point that wrt very specific guarantees, fully
> >> agree. In that sense we should just have it defined with the clang + __bpf__
> >> constraints mentioned earlier.
> >
> > Is this change going to happen?
> > I'm still having a compilation error when using master branch
>
> Yeah, I'll submit something tonight.
>
> Thanks,
> Daniel

Great, Thanks!

Yaniv