bpf: samples/bpf/sockex2: Assertion `setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd, sizeof(prog_fd[0])) == 0' failed.

bpf: samples/bpf/sockex2: Assertion `setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd, sizeof(prog_fd[0])) == 0' failed.

[Wang Sheng-Hui]

Sorry to trouble you!

I run samples/bpf/sockex2 failed with 
"Assertion `setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd, sizeof(prog_fd[0])) == 0' failed."

Then I run " strace ./sockex2" and got:
...
bind(3, {sa_family=AF_PACKET, sll_protocol=htons(ETH_P_ALL), sll_ifindex=if_nametoindex("lo"), sll_hatype=ARPHRD_NETROM, sll_pkttype=PACKET_HOST, sll_halen=0}, 20) = 0
setsockopt(3, SOL_SOCKET, SO_ATTACH_BPF, [0], 4) = -1 EINVAL (Invalid argument)
write(2, "sockex2: /root/linux/samples/bpf"..., 156sockex2: /root/linux/samples/bpf/sockex2_user.c:35: main: Assertion `setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd, sizeof(prog_fd[0])) == 0' failed.
) = 156
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb8ec4bf000
rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
getpid()                                = 3513
gettid()                                = 3513
tgkill(3513, 3513, SIGABRT)             = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=3513, si_uid=0} ---
+++ killed by SIGABRT +++
Aborted


I tried to find out the root cause but failed:
sockex2 is similar with sockex1, except it run with large bpf code and hash map.
Both can pass the program load phase and open raw socket, so I think there is nothing
wrong with memcharge and other checks.
While sockex1 can work, sockext2 failed with BPF attach to socket.
I read though the sock attach code path and didn't find any specific check to report failures. 

Can someone help to fix the failure, please?

Thanks,
shenghui

Re: bpf: samples/bpf/sockex2: Assertion `setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd, sizeof(prog_fd[0])) == 0' failed.

[Daniel Borkmann]

On 04/23/2018 04:56 AM, Wang Sheng-Hui wrote:
> Sorry to trouble you!
> 
> I run samples/bpf/sockex2 failed with 
> "Assertion `setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd, sizeof(prog_fd[0])) == 0' failed."
> 
> Then I run " strace ./sockex2" and got:
> ...
> bind(3, {sa_family=AF_PACKET, sll_protocol=htons(ETH_P_ALL), sll_ifindex=if_nametoindex("lo"), sll_hatype=ARPHRD_NETROM, sll_pkttype=PACKET_HOST, sll_halen=0}, 20) = 0
> setsockopt(3, SOL_SOCKET, SO_ATTACH_BPF, [0], 4) = -1 EINVAL (Invalid argument)
> write(2, "sockex2: /root/linux/samples/bpf"..., 156sockex2: /root/linux/samples/bpf/sockex2_user.c:35: main: Assertion `setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd, sizeof(prog_fd[0])) == 0' failed.
> ) = 156
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb8ec4bf000
> rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
> rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
> getpid()                                = 3513
> gettid()                                = 3513
> tgkill(3513, 3513, SIGABRT)             = 0
> rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=3513, si_uid=0} ---
> +++ killed by SIGABRT +++
> Aborted

[...]
bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_HASH, key_size=4, value_size=16, max_entries=1024}, 72) = 4
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_SOCKET_FILTER, insn_cnt=201, insns=0x2676cd0, license="GPL", log_level=0, log_size=0, log_buf=0, kern_version=0}, 72) = 5
close(3)                                = 0
socket(AF_PACKET, SOCK_RAW|SOCK_CLOEXEC|SOCK_NONBLOCK, 768) = 3
access("/proc/net", R_OK)               = 0
access("/proc/net/unix", R_OK)          = 0
socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 6
ioctl(6, SIOCGIFINDEX, {ifr_name="lo", }) = 0
close(6)                                = 0
bind(3, {sa_family=AF_PACKET, sll_protocol=htons(ETH_P_ALL), sll_ifindex=if_nametoindex("lo"), sll_hatype=ARPHRD_NETROM, sll_pkttype=PACKET_HOST, sll_halen=0}, 20) = 0
setsockopt(3, SOL_SOCKET, SO_ATTACH_BPF, [5], 4) = 0
pipe2([6, 7], O_CLOEXEC)                = 0
[...]

Works fine for me. The EINVAL in your case comes from the 'setsockopt(3, SOL_SOCKET,
SO_ATTACH_BPF, [0], 4)'. So you send fd 0 to SO_ATTACH_BPF, which is not a valid BPF
fd and therefore bails out here. You might want to debug bpf_load.c why it's failing
to load the program in your case, or check strace a bit further above where you do
the map and prog creation (as I copied in my case).

Cheers,
Daniel

Re: bpf: samples/bpf/sockex2: Assertion `setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd, sizeof(prog_fd[0])) == 0' failed.

[Daniel Borkmann]

On 04/24/2018 09:29 AM, shhuiw wrote:
> On 04/23/18 17:53, Daniel Borkmann wrote:
>> On 04/23/2018 04:56 AM, Wang Sheng-Hui wrote:
>>> Sorry to trouble you!
>>>
>>> I run samples/bpf/sockex2 failed with
>>> "Assertion `setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd, sizeof(prog_fd[0])) == 0' failed."
>>>
>>> Then I run " strace ./sockex2" and got:
>>> ...
>>> bind(3, {sa_family=AF_PACKET, sll_protocol=htons(ETH_P_ALL), sll_ifindex=if_nametoindex("lo"), sll_hatype=ARPHRD_NETROM, sll_pkttype=PACKET_HOST, sll_halen=0}, 20) = 0
>>> setsockopt(3, SOL_SOCKET, SO_ATTACH_BPF, [0], 4) = -1 EINVAL (Invalid argument)
>>> write(2, "sockex2: /root/linux/samples/bpf"..., 156sockex2: /root/linux/samples/bpf/sockex2_user.c:35: main: Assertion `setsockopt(sock, SOL_SOCKET, SO_ATTACH_BPF, prog_fd, sizeof(prog_fd[0])) == 0' failed.
>>> ) = 156
>>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb8ec4bf000
>>> rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
>>> rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
>>> getpid()                                = 3513
>>> gettid()                                = 3513
>>> tgkill(3513, 3513, SIGABRT)             = 0
>>> rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
>>> --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=3513, si_uid=0} ---
>>> +++ killed by SIGABRT +++
>>> Aborted
>> [...]
>> bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_HASH, key_size=4, value_size=16, max_entries=1024}, 72) = 4
>> bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_SOCKET_FILTER, insn_cnt=201, insns=0x2676cd0, license="GPL", log_level=0, log_size=0, log_buf=0, kern_version=0}, 72) = 5
>> close(3)                                = 0
>> socket(AF_PACKET, SOCK_RAW|SOCK_CLOEXEC|SOCK_NONBLOCK, 768) = 3
>> access("/proc/net", R_OK)               = 0
>> access("/proc/net/unix", R_OK)          = 0
>> socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 6
>> ioctl(6, SIOCGIFINDEX, {ifr_name="lo", }) = 0
>> close(6)                                = 0
>> bind(3, {sa_family=AF_PACKET, sll_protocol=htons(ETH_P_ALL), sll_ifindex=if_nametoindex("lo"), sll_hatype=ARPHRD_NETROM, sll_pkttype=PACKET_HOST, sll_halen=0}, 20) = 0
>> setsockopt(3, SOL_SOCKET, SO_ATTACH_BPF, [5], 4) = 0
>> pipe2([6, 7], O_CLOEXEC)                = 0
>> [...]
>>
>> Works fine for me. The EINVAL in your case comes from the 'setsockopt(3, SOL_SOCKET,
>> SO_ATTACH_BPF, [0], 4)'. So you send fd 0 to SO_ATTACH_BPF, which is not a valid BPF
>> fd and therefore bails out here. You might want to debug bpf_load.c why it's failing
>> to load the program in your case, or check strace a bit further above where you do
>> the map and prog creation (as I copied in my case).
>>
>> Cheers,
>> Daniel
>>
> Daniel,
> 
> I built the sample code by run
>      "make O=../buildkernel samples/bpf/ LLC=/usr/bin/llc clang=/usr/bin/clang"
> while llc & clang are provided by debian:
>     ~/buildkernel/samples/bpf# ls -l /usr/bin/llc
>     lrwxrwxrwx 1 root root 23 Dec  4 21:34 /usr/bin/llc -> ../lib/llvm-4.0/bin/llc
>     ~/buildkernel/samples/bpf# ls -l /usr/bin/clang
>     lrwxrwxrwx 1 root root 25 Dec  4 21:34 /usr/bin/clang -> ../lib/llvm-4.0/bin/clang
> ~/buildkernel/samples/bpf# llc --version
> LLVM (http://llvm.org/):
>   LLVM version 4.0.1
> ...
>   Registered Targets:
> ...
>     bpf        - BPF (host endian)
>     bpfeb      - BPF (big endian)
>     bpfel      - BPF (little endian)
> ...
> 
> There are 3 sockex BPF programs under samples/bpf, but only 'socket' section in sockex1_kern.o can be
> detected, sockex[23]_kern.o failed.
> ------------------------------------------------------------------------
> ~/buildkernel/samples/bpf# llvm-objdump -h ./sockex1_kern.o
> ./sockex1_kern.o:    file format ELF64-BPF
> Sections:
> Idx Name          Size      Address          Type
>   0               00000000 0000000000000000
>   1 .strtab       00000057 0000000000000000
>   2 .text         00000000 0000000000000000 TEXT DATA
>   3 socket1       00000078 0000000000000000 TEXT DATA
>   4 .relsocket1   00000010 0000000000000000
>   5 maps          0000001c 0000000000000000 DATA
>   6 license       00000004 0000000000000000 DATA
>   7 .eh_frame     00000028 0000000000000000 DATA
>   8 .rel.eh_frame 00000010 0000000000000000
>   9 .symtab       00000090 0000000000000000
> 
> ~/buildkernel/samples/bpf# llvm-objdump -h ./sockex2_kern.o
> ./sockex2_kern.o:    file format ELF64-BPF
> Sections:
> Idx Name          Size      Address          Type
>   0               00000000 0000000000000000
>   1 .strtab       00000017 0000000000000000
>   2 .text         00000000 0000000000000000 TEXT DATA
>   3 .symtab       00000018 0000000000000000
> 
> ~/buildkernel/samples/bpf# llvm-objdump -h ./sockex3_kern.o
> ./sockex3_kern.o:    file format ELF64-BPF
> Sections:
> Idx Name          Size      Address          Type
>   0               00000000 0000000000000000
>   1 .strtab       00000017 0000000000000000
>   2 .text         00000000 0000000000000000 TEXT DATA
>   3 .symtab       00000018 0000000000000000
> 
> 
> Do you know how to fix it, please?

Did you by any chance hit an error like the below during compilation? The first
sockex1_kern.o compiled fine for me from the kernel samples dir, but the second
sockex2_kern.o and subsequent bail out with "error: 'asm goto' constructs are not
supported yet":

# make
[...]
clang  -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/6.4.1/include -I./arch/x86/include -I./arch/x86/include/generated  -I./include -I./arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h  -I/home/foo/trees/bpf-next/samples/bpf \
	-I./tools/testing/selftests/bpf/ \
	-D__KERNEL__ -Wno-unused-value -Wno-pointer-sign \
	-D__TARGET_ARCH_x86 -Wno-compare-distinct-pointer-types \
	-Wno-gnu-variable-sized-type-not-at-end \
	-Wno-address-of-packed-member -Wno-tautological-compare \
	-Wno-unknown-warning-option  \
	-O2 -emit-llvm -c /home/foo/trees/bpf-next/samples/bpf/sockex1_kern.c -o -| llc -march=bpf -filetype=obj -o /home/foo/trees/bpf-next/samples/bpf/sockex1_kern.o
clang  -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/6.4.1/include -I./arch/x86/include -I./arch/x86/include/generated  -I./include -I./arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h  -I/home/foo/trees/bpf-next/samples/bpf \
	-I./tools/testing/selftests/bpf/ \
	-D__KERNEL__ -Wno-unused-value -Wno-pointer-sign \
	-D__TARGET_ARCH_x86 -Wno-compare-distinct-pointer-types \
	-Wno-gnu-variable-sized-type-not-at-end \
	-Wno-address-of-packed-member -Wno-tautological-compare \
	-Wno-unknown-warning-option  \
	-O2 -emit-llvm -c /home/foo/trees/bpf-next/samples/bpf/sockex2_kern.c -o -| llc -march=bpf -filetype=obj -o /home/foo/trees/bpf-next/samples/bpf/sockex2_kern.o
In file included from /home/foo/trees/bpf-next/samples/bpf/sockex2_kern.c:3:
In file included from ./include/uapi/linux/in.h:24:
In file included from ./include/linux/socket.h:8:
In file included from ./include/linux/uio.h:13:
In file included from ./include/linux/thread_info.h:38:
In file included from ./arch/x86/include/asm/thread_info.h:53:
./arch/x86/include/asm/cpufeature.h:150:2: error: 'asm goto' constructs are not supported yet
        asm_volatile_goto("1: jmp 6f\n"
        ^
./include/linux/compiler-gcc.h:290:42: note: expanded from macro 'asm_volatile_goto'
#define asm_volatile_goto(x...) do { asm goto(x); asm (""); } while (0)
                                         ^
1 error generated.
[...]

Which leads to:

# llvm-objdump -h ./sockex1_kern.o

./sockex1_kern.o:	file format ELF64-BPF

Sections:
Idx Name          Size      Address          Type
  0               00000000 0000000000000000
  1 .strtab       00000057 0000000000000000
  2 .text         00000000 0000000000000000 TEXT DATA
  3 socket1       00000078 0000000000000000 TEXT DATA
  4 .relsocket1   00000010 0000000000000000
  5 maps          0000001c 0000000000000000 DATA
  6 license       00000004 0000000000000000 DATA
  7 .eh_frame     00000028 0000000000000000 DATA
  8 .rel.eh_frame 00000010 0000000000000000
  9 .symtab       00000090 0000000000000000

But ...

# llvm-objdump -h ./sockex2_kern.o

./sockex2_kern.o:	file format ELF64-BPF

Sections:
Idx Name          Size      Address          Type
  0               00000000 0000000000000000
  1 .strtab       00000017 0000000000000000
  2 .text         00000000 0000000000000000 TEXT DATA
  3 .symtab       00000018 0000000000000000
#

There's a fix from Yonghong here:

  https://patchwork.kernel.org/patch/10341471/

Could you try with that patch?

Thanks,
Daniel