Cannot load eBPF program as XDP

Cannot load eBPF program as XDP

[MD I. Islam]

Hi

I'm using http://lingrok.org/xref/linux-net-next/samples/bpf/parse_simple.c.

I can load the program as tc_cls using following commands:

sudo tc qdisc add dev eno1 clsact
sudo tc filter add dev eno1  ingress bpf da obj parse_simple.o sec simple

But I get error when I load the same program in XDP:

sudo ip link set dev eno1 xdp obj parse_simple.o section simple

Prog section 'simple' rejected: Permission denied (13)!
 - Type:         6
 - Instructions: 23 (0 over limit)
 - License:      GPL

Verifier analysis:

0: (b7) r0 = 0
1: (61) r2 = *(u32 *)(r1 +80)
invalid bpf_context access off=80 size=4

Error fetching program/map!

I made sure that the parse_simple.o have necessary permission. Could
you please advise what is wrong here? I'm using Kernel 4.11+. Does XDP
need any specific kernel configuration enabled? clang and LLVM
versions are as following:

clang --version
clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
Target: x86_64-pc-linux-gnu

llc --version
LLVM (http://llvm.org/):
  LLVM version 3.8.0

  Optimized build.
  Built Jul  9 2016 (11:22:59).
  Default target: x86_64-pc-linux-gnu
  Host CPU: haswell

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    amdgcn     - AMD GCN GPUs
    arm        - ARM
    arm64      - ARM64 (little endian)
    armeb      - ARM (big endian)
    bpf        - BPF (host endian)
    bpfeb      - BPF (big endian)
    bpfel      - BPF (little endian)
    cpp        - C++ backend
    hexagon    - Hexagon
    mips       - Mips
    mips64     - Mips64 [experimental]
    mips64el   - Mips64el [experimental]
    mipsel     - Mipsel
    msp430     - MSP430 [experimental]
    nvptx      - NVIDIA PTX 32-bit
    nvptx64    - NVIDIA PTX 64-bit
    ppc32      - PowerPC 32
    ppc64      - PowerPC 64
    ppc64le    - PowerPC 64 LE
    r600       - AMD GPUs HD2XXX-HD6XXX
    sparc      - Sparc
    sparcel    - Sparc LE
    sparcv9    - Sparc V9
    systemz    - SystemZ
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64
    xcore      - XCore

Many thanks
Tamim

Re: Cannot load eBPF program as XDP

[Y Song]

xdp meta data is different:

struct xdp_md {
        __u32 data;
        __u32 data_end;
};

All other fields available for skb metadata won't be available in XDP.
That is way
you get verification failure.

On Tue, Jun 27, 2017 at 7:00 PM, MD I. Islam <tamim@csebuet.org> wrote:
> Hi
>
> I'm using http://lingrok.org/xref/linux-net-next/samples/bpf/parse_simple.c.
>
> I can load the program as tc_cls using following commands:
>
> sudo tc qdisc add dev eno1 clsact
> sudo tc filter add dev eno1  ingress bpf da obj parse_simple.o sec simple
>
> But I get error when I load the same program in XDP:
>
> sudo ip link set dev eno1 xdp obj parse_simple.o section simple
>
> Prog section 'simple' rejected: Permission denied (13)!
>  - Type:         6
>  - Instructions: 23 (0 over limit)
>  - License:      GPL
>
> Verifier analysis:
>
> 0: (b7) r0 = 0
> 1: (61) r2 = *(u32 *)(r1 +80)
> invalid bpf_context access off=80 size=4
>
> Error fetching program/map!
>
> I made sure that the parse_simple.o have necessary permission. Could
> you please advise what is wrong here? I'm using Kernel 4.11+. Does XDP
> need any specific kernel configuration enabled? clang and LLVM
> versions are as following:
>
> clang --version
> clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
> Target: x86_64-pc-linux-gnu
>
> llc --version
> LLVM (http://llvm.org/):
>   LLVM version 3.8.0
>
>   Optimized build.
>   Built Jul  9 2016 (11:22:59).
>   Default target: x86_64-pc-linux-gnu
>   Host CPU: haswell
>
>   Registered Targets:
>     aarch64    - AArch64 (little endian)
>     aarch64_be - AArch64 (big endian)
>     amdgcn     - AMD GCN GPUs
>     arm        - ARM
>     arm64      - ARM64 (little endian)
>     armeb      - ARM (big endian)
>     bpf        - BPF (host endian)
>     bpfeb      - BPF (big endian)
>     bpfel      - BPF (little endian)
>     cpp        - C++ backend
>     hexagon    - Hexagon
>     mips       - Mips
>     mips64     - Mips64 [experimental]
>     mips64el   - Mips64el [experimental]
>     mipsel     - Mipsel
>     msp430     - MSP430 [experimental]
>     nvptx      - NVIDIA PTX 32-bit
>     nvptx64    - NVIDIA PTX 64-bit
>     ppc32      - PowerPC 32
>     ppc64      - PowerPC 64
>     ppc64le    - PowerPC 64 LE
>     r600       - AMD GPUs HD2XXX-HD6XXX
>     sparc      - Sparc
>     sparcel    - Sparc LE
>     sparcv9    - Sparc V9
>     systemz    - SystemZ
>     thumb      - Thumb
>     thumbeb    - Thumb (big endian)
>     x86        - 32-bit X86: Pentium-Pro and above
>     x86-64     - 64-bit X86: EM64T and AMD64
>     xcore      - XCore
>
> Many thanks
> Tamim

Re: Cannot load eBPF program as XDP

[MD I. Islam]

Hi Song

On Tue, Jun 27, 2017 at 10:05 PM, Y Song <ys114321@gmail.com> wrote:
> xdp meta data is different:
>
> struct xdp_md {
>         __u32 data;
>         __u32 data_end;
> };
>
> All other fields available for skb metadata won't be available in XDP.
> That is way
> you get verification failure.
>


Thanks for the quick reply! Now I'm trying xdp1_kern.c which uses
xdp_md. Now I'm getting the error:

Note: 8 bytes struct bpf_elf_map fixup performed due to size mismatch!
RTNETLINK answers: Operation not supported

Could you please advise me what is wrong here?

> On Tue, Jun 27, 2017 at 7:00 PM, MD I. Islam <tamim@csebuet.org> wrote:
>> Hi
>>
>> I'm using http://lingrok.org/xref/linux-net-next/samples/bpf/parse_simple.c.
>>
>> I can load the program as tc_cls using following commands:
>>
>> sudo tc qdisc add dev eno1 clsact
>> sudo tc filter add dev eno1  ingress bpf da obj parse_simple.o sec simple
>>
>> But I get error when I load the same program in XDP:
>>
>> sudo ip link set dev eno1 xdp obj parse_simple.o section simple
>>
>> Prog section 'simple' rejected: Permission denied (13)!
>>  - Type:         6
>>  - Instructions: 23 (0 over limit)
>>  - License:      GPL
>>
>> Verifier analysis:
>>
>> 0: (b7) r0 = 0
>> 1: (61) r2 = *(u32 *)(r1 +80)
>> invalid bpf_context access off=80 size=4
>>
>> Error fetching program/map!
>>
>> I made sure that the parse_simple.o have necessary permission. Could
>> you please advise what is wrong here? I'm using Kernel 4.11+. Does XDP
>> need any specific kernel configuration enabled? clang and LLVM
>> versions are as following:
>>
>> clang --version
>> clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
>> Target: x86_64-pc-linux-gnu
>>
>> llc --version
>> LLVM (http://llvm.org/):
>>   LLVM version 3.8.0
>>
>>   Optimized build.
>>   Built Jul  9 2016 (11:22:59).
>>   Default target: x86_64-pc-linux-gnu
>>   Host CPU: haswell
>>
>>   Registered Targets:
>>     aarch64    - AArch64 (little endian)
>>     aarch64_be - AArch64 (big endian)
>>     amdgcn     - AMD GCN GPUs
>>     arm        - ARM
>>     arm64      - ARM64 (little endian)
>>     armeb      - ARM (big endian)
>>     bpf        - BPF (host endian)
>>     bpfeb      - BPF (big endian)
>>     bpfel      - BPF (little endian)
>>     cpp        - C++ backend
>>     hexagon    - Hexagon
>>     mips       - Mips
>>     mips64     - Mips64 [experimental]
>>     mips64el   - Mips64el [experimental]
>>     mipsel     - Mipsel
>>     msp430     - MSP430 [experimental]
>>     nvptx      - NVIDIA PTX 32-bit
>>     nvptx64    - NVIDIA PTX 64-bit
>>     ppc32      - PowerPC 32
>>     ppc64      - PowerPC 64
>>     ppc64le    - PowerPC 64 LE
>>     r600       - AMD GPUs HD2XXX-HD6XXX
>>     sparc      - Sparc
>>     sparcel    - Sparc LE
>>     sparcv9    - Sparc V9
>>     systemz    - SystemZ
>>     thumb      - Thumb
>>     thumbeb    - Thumb (big endian)
>>     x86        - 32-bit X86: Pentium-Pro and above
>>     x86-64     - 64-bit X86: EM64T and AMD64
>>     xcore      - XCore
>>
>> Many thanks
>> Tamim

Thanks
Tamim

Re: Cannot load eBPF program as XDP

[Y Song]

On Tue, Jun 27, 2017 at 7:32 PM, MD I. Islam <tamim@csebuet.org> wrote:
> Hi Song
>
> On Tue, Jun 27, 2017 at 10:05 PM, Y Song <ys114321@gmail.com> wrote:
>> xdp meta data is different:
>>
>> struct xdp_md {
>>         __u32 data;
>>         __u32 data_end;
>> };
>>
>> All other fields available for skb metadata won't be available in XDP.
>> That is way
>> you get verification failure.
>>
>
>
> Thanks for the quick reply! Now I'm trying xdp1_kern.c which uses
> xdp_md. Now I'm getting the error:
>
> Note: 8 bytes struct bpf_elf_map fixup performed due to size mismatch!
> RTNETLINK answers: Operation not supported

I do not know what "Note: 8 bytes ..." error means, I did not find a
similar error
message pattern in latest net-next.

For RTNETLINK error, most like the networking interface you tried to attach
does not support XDP. It needs to a physical device or a generic
device, based on options. Check the user space program and you can
find various options.

>
> Could you please advise me what is wrong here?
>
>> On Tue, Jun 27, 2017 at 7:00 PM, MD I. Islam <tamim@csebuet.org> wrote:
>>> Hi
>>>
>>> I'm using http://lingrok.org/xref/linux-net-next/samples/bpf/parse_simple.c.
>>>
>>> I can load the program as tc_cls using following commands:
>>>
>>> sudo tc qdisc add dev eno1 clsact
>>> sudo tc filter add dev eno1  ingress bpf da obj parse_simple.o sec simple
>>>
>>> But I get error when I load the same program in XDP:
>>>
>>> sudo ip link set dev eno1 xdp obj parse_simple.o section simple
>>>
>>> Prog section 'simple' rejected: Permission denied (13)!
>>>  - Type:         6
>>>  - Instructions: 23 (0 over limit)
>>>  - License:      GPL
>>>
>>> Verifier analysis:
>>>
>>> 0: (b7) r0 = 0
>>> 1: (61) r2 = *(u32 *)(r1 +80)
>>> invalid bpf_context access off=80 size=4
>>>
>>> Error fetching program/map!
>>>
>>> I made sure that the parse_simple.o have necessary permission. Could
>>> you please advise what is wrong here? I'm using Kernel 4.11+. Does XDP
>>> need any specific kernel configuration enabled? clang and LLVM
>>> versions are as following:
>>>
>>> clang --version
>>> clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
>>> Target: x86_64-pc-linux-gnu
>>>
>>> llc --version
>>> LLVM (http://llvm.org/):
>>>   LLVM version 3.8.0
>>>
>>>   Optimized build.
>>>   Built Jul  9 2016 (11:22:59).
>>>   Default target: x86_64-pc-linux-gnu
>>>   Host CPU: haswell
>>>
>>>   Registered Targets:
>>>     aarch64    - AArch64 (little endian)
>>>     aarch64_be - AArch64 (big endian)
>>>     amdgcn     - AMD GCN GPUs
>>>     arm        - ARM
>>>     arm64      - ARM64 (little endian)
>>>     armeb      - ARM (big endian)
>>>     bpf        - BPF (host endian)
>>>     bpfeb      - BPF (big endian)
>>>     bpfel      - BPF (little endian)
>>>     cpp        - C++ backend
>>>     hexagon    - Hexagon
>>>     mips       - Mips
>>>     mips64     - Mips64 [experimental]
>>>     mips64el   - Mips64el [experimental]
>>>     mipsel     - Mipsel
>>>     msp430     - MSP430 [experimental]
>>>     nvptx      - NVIDIA PTX 32-bit
>>>     nvptx64    - NVIDIA PTX 64-bit
>>>     ppc32      - PowerPC 32
>>>     ppc64      - PowerPC 64
>>>     ppc64le    - PowerPC 64 LE
>>>     r600       - AMD GPUs HD2XXX-HD6XXX
>>>     sparc      - Sparc
>>>     sparcel    - Sparc LE
>>>     sparcv9    - Sparc V9
>>>     systemz    - SystemZ
>>>     thumb      - Thumb
>>>     thumbeb    - Thumb (big endian)
>>>     x86        - 32-bit X86: Pentium-Pro and above
>>>     x86-64     - 64-bit X86: EM64T and AMD64
>>>     xcore      - XCore
>>>
>>> Many thanks
>>> Tamim
>
> Thanks
> Tamim

Re: Cannot load eBPF program as XDP

[Daniel Borkmann]

On 06/28/2017 04:32 AM, MD I. Islam wrote:
> On Tue, Jun 27, 2017 at 10:05 PM, Y Song <ys114321@gmail.com> wrote:
>> xdp meta data is different:
>>
>> struct xdp_md {
>>          __u32 data;
>>          __u32 data_end;
>> };
>>
>> All other fields available for skb metadata won't be available in XDP.
>> That is way
>> you get verification failure.
>
> Thanks for the quick reply! Now I'm trying xdp1_kern.c which uses
> xdp_md. Now I'm getting the error:
>
> Note: 8 bytes struct bpf_elf_map fixup performed due to size mismatch!

This one is harmless, just telling you that the native bpf_elf_map
from iproute2 is different than the one you're loading, so it fixes
up the remaining 8 bytes automatically.

> RTNETLINK answers: Operation not supported

This one seems likely that the driver you're trying to attach this to
does not have XDP support. What's the driver you're using?

> Could you please advise me what is wrong here?
>
>> On Tue, Jun 27, 2017 at 7:00 PM, MD I. Islam <tamim@csebuet.org> wrote:
>>> Hi
>>>
>>> I'm using http://lingrok.org/xref/linux-net-next/samples/bpf/parse_simple.c.
>>>
>>> I can load the program as tc_cls using following commands:
>>>
>>> sudo tc qdisc add dev eno1 clsact
>>> sudo tc filter add dev eno1  ingress bpf da obj parse_simple.o sec simple
>>>
>>> But I get error when I load the same program in XDP:
>>>
>>> sudo ip link set dev eno1 xdp obj parse_simple.o section simple
>>>
>>> Prog section 'simple' rejected: Permission denied (13)!
>>>   - Type:         6
>>>   - Instructions: 23 (0 over limit)
>>>   - License:      GPL
>>>
>>> Verifier analysis:
>>>
>>> 0: (b7) r0 = 0
>>> 1: (61) r2 = *(u32 *)(r1 +80)
>>> invalid bpf_context access off=80 size=4
>>>
>>> Error fetching program/map!
>>>
>>> I made sure that the parse_simple.o have necessary permission. Could
>>> you please advise what is wrong here? I'm using Kernel 4.11+. Does XDP
>>> need any specific kernel configuration enabled? clang and LLVM
>>> versions are as following:
>>>
>>> clang --version
>>> clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
>>> Target: x86_64-pc-linux-gnu
>>>
>>> llc --version
>>> LLVM (http://llvm.org/):
>>>    LLVM version 3.8.0
>>>
>>>    Optimized build.
>>>    Built Jul  9 2016 (11:22:59).
>>>    Default target: x86_64-pc-linux-gnu
>>>    Host CPU: haswell
>>>
>>>    Registered Targets:
>>>      aarch64    - AArch64 (little endian)
>>>      aarch64_be - AArch64 (big endian)
>>>      amdgcn     - AMD GCN GPUs
>>>      arm        - ARM
>>>      arm64      - ARM64 (little endian)
>>>      armeb      - ARM (big endian)
>>>      bpf        - BPF (host endian)
>>>      bpfeb      - BPF (big endian)
>>>      bpfel      - BPF (little endian)
>>>      cpp        - C++ backend
>>>      hexagon    - Hexagon
>>>      mips       - Mips
>>>      mips64     - Mips64 [experimental]
>>>      mips64el   - Mips64el [experimental]
>>>      mipsel     - Mipsel
>>>      msp430     - MSP430 [experimental]
>>>      nvptx      - NVIDIA PTX 32-bit
>>>      nvptx64    - NVIDIA PTX 64-bit
>>>      ppc32      - PowerPC 32
>>>      ppc64      - PowerPC 64
>>>      ppc64le    - PowerPC 64 LE
>>>      r600       - AMD GPUs HD2XXX-HD6XXX
>>>      sparc      - Sparc
>>>      sparcel    - Sparc LE
>>>      sparcv9    - Sparc V9
>>>      systemz    - SystemZ
>>>      thumb      - Thumb
>>>      thumbeb    - Thumb (big endian)
>>>      x86        - 32-bit X86: Pentium-Pro and above
>>>      x86-64     - 64-bit X86: EM64T and AMD64
>>>      xcore      - XCore
>>>
>>> Many thanks
>>> Tamim
>
> Thanks
> Tamim
>

Re: Cannot load eBPF program as XDP

[Jesper Dangaard Brouer]

On Tue, 27 Jun 2017 22:32:26 -0400 "MD I. Islam" <tamim@csebuet.org> wrote:

> Hi Song
> 
> On Tue, Jun 27, 2017 at 10:05 PM, Y Song <ys114321@gmail.com> wrote:
> > xdp meta data is different:
> >
> > struct xdp_md {
> >         __u32 data;
> >         __u32 data_end;
> > };
> >
> > All other fields available for skb metadata won't be available in XDP.
> > That is way
> > you get verification failure.
> >  
> 
> 
> Thanks for the quick reply! Now I'm trying xdp1_kern.c which uses
> xdp_md. Now I'm getting the error:
> 
> Note: 8 bytes struct bpf_elf_map fixup performed due to size mismatch!
> RTNETLINK answers: Operation not supported
> 
> Could you please advise me what is wrong here?

The xdp1_kern.c bpf program expect to be loaded with the tool
xdp1_user.c (compiled to executable xdp1) which uses the bpf-elf loader
located in the kernel/samples/bpf/ directory.

You are trying to load this xdp1_kern.c program with the TC utility,
which have it's own bpf-elf loader.  These two loaders are not expected
to be compatible... specifically struct bpf_elf_map are different.


p.s. I'm maintaining some bpf samples here: 
 https://github.com/netoptimizer/prototype-kernel/tree/master/kernel/samples/bpf

--Jesper
 
> > On Tue, Jun 27, 2017 at 7:00 PM, MD I. Islam <tamim@csebuet.org> wrote:  
> >> Hi
> >>
> >> I'm using http://lingrok.org/xref/linux-net-next/samples/bpf/parse_simple.c.
> >>
> >> I can load the program as tc_cls using following commands:
> >>
> >> sudo tc qdisc add dev eno1 clsact
> >> sudo tc filter add dev eno1  ingress bpf da obj parse_simple.o sec simple
> >>
> >> But I get error when I load the same program in XDP:
> >>
> >> sudo ip link set dev eno1 xdp obj parse_simple.o section simple
> >>
> >> Prog section 'simple' rejected: Permission denied (13)!
> >>  - Type:         6
> >>  - Instructions: 23 (0 over limit)
> >>  - License:      GPL
> >>
> >> Verifier analysis:
> >>
> >> 0: (b7) r0 = 0
> >> 1: (61) r2 = *(u32 *)(r1 +80)
> >> invalid bpf_context access off=80 size=4
> >>
> >> Error fetching program/map!
> >>
> >> I made sure that the parse_simple.o have necessary permission. Could
> >> you please advise what is wrong here? I'm using Kernel 4.11+. Does XDP
> >> need any specific kernel configuration enabled? clang and LLVM
> >> versions are as following:
> >>
> >> clang --version
> >> clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
> >> Target: x86_64-pc-linux-gnu
> >>
> >> llc --version
> >> LLVM (http://llvm.org/):
> >>   LLVM version 3.8.0
> >>
> >>   Optimized build.
> >>   Built Jul  9 2016 (11:22:59).
> >>   Default target: x86_64-pc-linux-gnu
> >>   Host CPU: haswell
> >>
> >>   Registered Targets:
> >>     aarch64    - AArch64 (little endian)
> >>     aarch64_be - AArch64 (big endian)
> >>     amdgcn     - AMD GCN GPUs
> >>     arm        - ARM
> >>     arm64      - ARM64 (little endian)
> >>     armeb      - ARM (big endian)
> >>     bpf        - BPF (host endian)
> >>     bpfeb      - BPF (big endian)
> >>     bpfel      - BPF (little endian)
> >>     cpp        - C++ backend
> >>     hexagon    - Hexagon
> >>     mips       - Mips
> >>     mips64     - Mips64 [experimental]
> >>     mips64el   - Mips64el [experimental]
> >>     mipsel     - Mipsel
> >>     msp430     - MSP430 [experimental]
> >>     nvptx      - NVIDIA PTX 32-bit
> >>     nvptx64    - NVIDIA PTX 64-bit
> >>     ppc32      - PowerPC 32
> >>     ppc64      - PowerPC 64
> >>     ppc64le    - PowerPC 64 LE
> >>     r600       - AMD GPUs HD2XXX-HD6XXX
> >>     sparc      - Sparc
> >>     sparcel    - Sparc LE
> >>     sparcv9    - Sparc V9
> >>     systemz    - SystemZ
> >>     thumb      - Thumb
> >>     thumbeb    - Thumb (big endian)
> >>     x86        - 32-bit X86: Pentium-Pro and above
> >>     x86-64     - 64-bit X86: EM64T and AMD64
> >>     xcore      - XCore
> >>
> >> Many thanks
> >> Tamim  
> 
> Thanks
> Tamim



-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer

Re: Cannot load eBPF program as XDP

[MD I. Islam]

Hi Daniel

On Wed, Jun 28, 2017 at 5:04 AM, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 06/28/2017 04:32 AM, MD I. Islam wrote:
>>
>> On Tue, Jun 27, 2017 at 10:05 PM, Y Song <ys114321@gmail.com> wrote:
>>>
>>> xdp meta data is different:
>>>
>>> struct xdp_md {
>>>          __u32 data;
>>>          __u32 data_end;
>>> };
>>>
>>> All other fields available for skb metadata won't be available in XDP.
>>> That is way
>>> you get verification failure.
>>
>>
>> Thanks for the quick reply! Now I'm trying xdp1_kern.c which uses
>> xdp_md. Now I'm getting the error:
>>
>> Note: 8 bytes struct bpf_elf_map fixup performed due to size mismatch!
>
>
> This one is harmless, just telling you that the native bpf_elf_map
> from iproute2 is different than the one you're loading, so it fixes
> up the remaining 8 bytes automatically.
>
>> RTNETLINK answers: Operation not supported
>
>
> This one seems likely that the driver you're trying to attach this to
> does not have XDP support. What's the driver you're using?

Yes, I was using RTL8111 driver which doesn't support XDP. I will
eventually use Netronome NIC which will hopefully work.

I'm facing the same problem in Mininet. Is there a way to use XDP in
Mininet? I am using Linux router in Mininet and I want to attach XDP
programs to virtual interfaces.

>
>
>> Could you please advise me what is wrong here?
>>
>>> On Tue, Jun 27, 2017 at 7:00 PM, MD I. Islam <tamim@csebuet.org> wrote:
>>>>
>>>> Hi
>>>>
>>>> I'm using
>>>> http://lingrok.org/xref/linux-net-next/samples/bpf/parse_simple.c.
>>>>
>>>> I can load the program as tc_cls using following commands:
>>>>
>>>> sudo tc qdisc add dev eno1 clsact
>>>> sudo tc filter add dev eno1  ingress bpf da obj parse_simple.o sec
>>>> simple
>>>>
>>>> But I get error when I load the same program in XDP:
>>>>
>>>> sudo ip link set dev eno1 xdp obj parse_simple.o section simple
>>>>
>>>> Prog section 'simple' rejected: Permission denied (13)!
>>>>   - Type:         6
>>>>   - Instructions: 23 (0 over limit)
>>>>   - License:      GPL
>>>>
>>>> Verifier analysis:
>>>>
>>>> 0: (b7) r0 = 0
>>>> 1: (61) r2 = *(u32 *)(r1 +80)
>>>> invalid bpf_context access off=80 size=4
>>>>
>>>> Error fetching program/map!
>>>>
>>>> I made sure that the parse_simple.o have necessary permission. Could
>>>> you please advise what is wrong here? I'm using Kernel 4.11+. Does XDP
>>>> need any specific kernel configuration enabled? clang and LLVM
>>>> versions are as following:
>>>>
>>>> clang --version
>>>> clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
>>>> Target: x86_64-pc-linux-gnu
>>>>
>>>> llc --version
>>>> LLVM (http://llvm.org/):
>>>>    LLVM version 3.8.0
>>>>
>>>>    Optimized build.
>>>>    Built Jul  9 2016 (11:22:59).
>>>>    Default target: x86_64-pc-linux-gnu
>>>>    Host CPU: haswell
>>>>
>>>>    Registered Targets:
>>>>      aarch64    - AArch64 (little endian)
>>>>      aarch64_be - AArch64 (big endian)
>>>>      amdgcn     - AMD GCN GPUs
>>>>      arm        - ARM
>>>>      arm64      - ARM64 (little endian)
>>>>      armeb      - ARM (big endian)
>>>>      bpf        - BPF (host endian)
>>>>      bpfeb      - BPF (big endian)
>>>>      bpfel      - BPF (little endian)
>>>>      cpp        - C++ backend
>>>>      hexagon    - Hexagon
>>>>      mips       - Mips
>>>>      mips64     - Mips64 [experimental]
>>>>      mips64el   - Mips64el [experimental]
>>>>      mipsel     - Mipsel
>>>>      msp430     - MSP430 [experimental]
>>>>      nvptx      - NVIDIA PTX 32-bit
>>>>      nvptx64    - NVIDIA PTX 64-bit
>>>>      ppc32      - PowerPC 32
>>>>      ppc64      - PowerPC 64
>>>>      ppc64le    - PowerPC 64 LE
>>>>      r600       - AMD GPUs HD2XXX-HD6XXX
>>>>      sparc      - Sparc
>>>>      sparcel    - Sparc LE
>>>>      sparcv9    - Sparc V9
>>>>      systemz    - SystemZ
>>>>      thumb      - Thumb
>>>>      thumbeb    - Thumb (big endian)
>>>>      x86        - 32-bit X86: Pentium-Pro and above
>>>>      x86-64     - 64-bit X86: EM64T and AMD64
>>>>      xcore      - XCore
>>>>
>>>> Many thanks
>>>> Tamim
>>
>>
>> Thanks
>> Tamim
>>
>

Re: Cannot load eBPF program as XDP

[MD I. Islam]

Hi Jesper

On Wed, Jun 28, 2017 at 4:26 PM, Jesper Dangaard Brouer
<brouer@redhat.com> wrote:
>
> On Tue, 27 Jun 2017 22:32:26 -0400 "MD I. Islam" <tamim@csebuet.org> wrote:
>
>> Hi Song
>>
>> On Tue, Jun 27, 2017 at 10:05 PM, Y Song <ys114321@gmail.com> wrote:
>> > xdp meta data is different:
>> >
>> > struct xdp_md {
>> >         __u32 data;
>> >         __u32 data_end;
>> > };
>> >
>> > All other fields available for skb metadata won't be available in XDP.
>> > That is way
>> > you get verification failure.
>> >
>>
>>
>> Thanks for the quick reply! Now I'm trying xdp1_kern.c which uses
>> xdp_md. Now I'm getting the error:
>>
>> Note: 8 bytes struct bpf_elf_map fixup performed due to size mismatch!
>> RTNETLINK answers: Operation not supported
>>
>> Could you please advise me what is wrong here?
>
> The xdp1_kern.c bpf program expect to be loaded with the tool
> xdp1_user.c (compiled to executable xdp1) which uses the bpf-elf loader
> located in the kernel/samples/bpf/ directory.
>
> You are trying to load this xdp1_kern.c program with the TC utility,
> which have it's own bpf-elf loader.  These two loaders are not expected
> to be compatible... specifically struct bpf_elf_map are different.
>

Thanks for pointing that out! I will use xdp1 executable to load the
BPF program. Currently my development machine doesn't have a XDP
compatible NIC. Is there a virtual NIC that I can use to prototype
XDP?

>
> p.s. I'm maintaining some bpf samples here:
>  https://github.com/netoptimizer/prototype-kernel/tree/master/kernel/samples/bpf
>
> --Jesper
>
>> > On Tue, Jun 27, 2017 at 7:00 PM, MD I. Islam <tamim@csebuet.org> wrote:
>> >> Hi
>> >>
>> >> I'm using http://lingrok.org/xref/linux-net-next/samples/bpf/parse_simple.c.
>> >>
>> >> I can load the program as tc_cls using following commands:
>> >>
>> >> sudo tc qdisc add dev eno1 clsact
>> >> sudo tc filter add dev eno1  ingress bpf da obj parse_simple.o sec simple
>> >>
>> >> But I get error when I load the same program in XDP:
>> >>
>> >> sudo ip link set dev eno1 xdp obj parse_simple.o section simple
>> >>
>> >> Prog section 'simple' rejected: Permission denied (13)!
>> >>  - Type:         6
>> >>  - Instructions: 23 (0 over limit)
>> >>  - License:      GPL
>> >>
>> >> Verifier analysis:
>> >>
>> >> 0: (b7) r0 = 0
>> >> 1: (61) r2 = *(u32 *)(r1 +80)
>> >> invalid bpf_context access off=80 size=4
>> >>
>> >> Error fetching program/map!
>> >>
>> >> I made sure that the parse_simple.o have necessary permission. Could
>> >> you please advise what is wrong here? I'm using Kernel 4.11+. Does XDP
>> >> need any specific kernel configuration enabled? clang and LLVM
>> >> versions are as following:
>> >>
>> >> clang --version
>> >> clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
>> >> Target: x86_64-pc-linux-gnu
>> >>
>> >> llc --version
>> >> LLVM (http://llvm.org/):
>> >>   LLVM version 3.8.0
>> >>
>> >>   Optimized build.
>> >>   Built Jul  9 2016 (11:22:59).
>> >>   Default target: x86_64-pc-linux-gnu
>> >>   Host CPU: haswell
>> >>
>> >>   Registered Targets:
>> >>     aarch64    - AArch64 (little endian)
>> >>     aarch64_be - AArch64 (big endian)
>> >>     amdgcn     - AMD GCN GPUs
>> >>     arm        - ARM
>> >>     arm64      - ARM64 (little endian)
>> >>     armeb      - ARM (big endian)
>> >>     bpf        - BPF (host endian)
>> >>     bpfeb      - BPF (big endian)
>> >>     bpfel      - BPF (little endian)
>> >>     cpp        - C++ backend
>> >>     hexagon    - Hexagon
>> >>     mips       - Mips
>> >>     mips64     - Mips64 [experimental]
>> >>     mips64el   - Mips64el [experimental]
>> >>     mipsel     - Mipsel
>> >>     msp430     - MSP430 [experimental]
>> >>     nvptx      - NVIDIA PTX 32-bit
>> >>     nvptx64    - NVIDIA PTX 64-bit
>> >>     ppc32      - PowerPC 32
>> >>     ppc64      - PowerPC 64
>> >>     ppc64le    - PowerPC 64 LE
>> >>     r600       - AMD GPUs HD2XXX-HD6XXX
>> >>     sparc      - Sparc
>> >>     sparcel    - Sparc LE
>> >>     sparcv9    - Sparc V9
>> >>     systemz    - SystemZ
>> >>     thumb      - Thumb
>> >>     thumbeb    - Thumb (big endian)
>> >>     x86        - 32-bit X86: Pentium-Pro and above
>> >>     x86-64     - 64-bit X86: EM64T and AMD64
>> >>     xcore      - XCore
>> >>
>> >> Many thanks
>> >> Tamim
>>
>> Thanks
>> Tamim
>
>
>
> --
> Best regards,
>   Jesper Dangaard Brouer
>   MSc.CS, Principal Kernel Engineer at Red Hat
>   LinkedIn: http://www.linkedin.com/in/brouer

Many thanks
Tamim

Re: Cannot load eBPF program as XDP

[Daniel Borkmann]

On 06/29/2017 12:40 AM, MD I. Islam wrote:
> On Wed, Jun 28, 2017 at 5:04 AM, Daniel Borkmann <daniel@iogearbox.net> wrote:
>> On 06/28/2017 04:32 AM, MD I. Islam wrote:
>>> On Tue, Jun 27, 2017 at 10:05 PM, Y Song <ys114321@gmail.com> wrote:
>>>>
>>>> xdp meta data is different:
>>>>
>>>> struct xdp_md {
>>>>           __u32 data;
>>>>           __u32 data_end;
>>>> };
>>>>
>>>> All other fields available for skb metadata won't be available in XDP.
>>>> That is way
>>>> you get verification failure.
>>>
>>> Thanks for the quick reply! Now I'm trying xdp1_kern.c which uses
>>> xdp_md. Now I'm getting the error:
>>>
>>> Note: 8 bytes struct bpf_elf_map fixup performed due to size mismatch!
>>
>> This one is harmless, just telling you that the native bpf_elf_map
>> from iproute2 is different than the one you're loading, so it fixes
>> up the remaining 8 bytes automatically.
>>
>>> RTNETLINK answers: Operation not supported
>>
>> This one seems likely that the driver you're trying to attach this to
>> does not have XDP support. What's the driver you're using?
>
> Yes, I was using RTL8111 driver which doesn't support XDP. I will
> eventually use Netronome NIC which will hopefully work.
>
> I'm facing the same problem in Mininet. Is there a way to use XDP in
> Mininet? I am using Linux router in Mininet and I want to attach XDP
> programs to virtual interfaces.

You can use generic XDP, which will load also with drivers not
supporting XDP natively (the fallback to generic XDP is transparent).
It's part of 4.12 kernel.

Re: Cannot load eBPF program as XDP

[Jesper Dangaard Brouer]

On Thu, 29 Jun 2017 02:19:57 +0200
Daniel Borkmann <daniel@iogearbox.net> wrote:

> On 06/29/2017 12:40 AM, MD I. Islam wrote:
> > On Wed, Jun 28, 2017 at 5:04 AM, Daniel Borkmann <daniel@iogearbox.net> wrote:  
> >> On 06/28/2017 04:32 AM, MD I. Islam wrote:  
> >>> On Tue, Jun 27, 2017 at 10:05 PM, Y Song <ys114321@gmail.com> wrote:  
> >>>>
> >>>> xdp meta data is different:
> >>>>
> >>>> struct xdp_md {
> >>>>           __u32 data;
> >>>>           __u32 data_end;
> >>>> };
> >>>>
> >>>> All other fields available for skb metadata won't be available in XDP.
> >>>> That is way
> >>>> you get verification failure.  
> >>>
> >>> Thanks for the quick reply! Now I'm trying xdp1_kern.c which uses
> >>> xdp_md. Now I'm getting the error:
> >>>
> >>> Note: 8 bytes struct bpf_elf_map fixup performed due to size mismatch!  
> >>
> >> This one is harmless, just telling you that the native bpf_elf_map
> >> from iproute2 is different than the one you're loading, so it fixes
> >> up the remaining 8 bytes automatically.
> >>  
> >>> RTNETLINK answers: Operation not supported  
> >>
> >> This one seems likely that the driver you're trying to attach this to
> >> does not have XDP support. What's the driver you're using?  
> >
> > Yes, I was using RTL8111 driver which doesn't support XDP. I will
> > eventually use Netronome NIC which will hopefully work.
> >
> > I'm facing the same problem in Mininet. Is there a way to use XDP in
> > Mininet? I am using Linux router in Mininet and I want to attach XDP
> > programs to virtual interfaces.  
> 
> You can use generic XDP, which will load also with drivers not
> supporting XDP natively (the fallback to generic XDP is transparent).
> It's part of 4.12 kernel.

Yes, exactly. With kernel 4.12 you can develop your XDP program, before
you get your hands on a NIC what have implemented the XDP eBPF-hook.

It is slower, compared here:
https://prototype-kernel.readthedocs.io/en/latest/blogposts/xdp25_eval_generic_xdp_tx.html

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer