[PATCH bpf-next 0/3] selftests: bpf: improve bpf object file rules

[PATCH bpf-next 0/3] selftests: bpf: improve bpf object file rules

[Jiong Wang]

This set improves bpf object file related rules in selftests Makefile.
  - tell git to ignore the build dir "alu32".
  - extend sub-register mode compilation to all bpf object files to give
    LLVM compiler bpf back-end more exercise.
  - auto-generate bpf kernel object file list.

Jiong Wang (3):
  selftests: bpf: add "alu32" to .gitignore
  selftests: bpf: extend sub-register mode compilation to all bpf object
    files
  selftests: bpf: centre kernel bpf objects under new subdir
    "kern_progs"

 tools/testing/selftests/bpf/.gitignore             |  1 +
 tools/testing/selftests/bpf/Makefile               | 35 +++++-----------------
 .../selftests/bpf/{ => kern_progs}/bpf_flow.c      |  0
 .../selftests/bpf/{ => kern_progs}/connect4_prog.c |  0
 .../selftests/bpf/{ => kern_progs}/connect6_prog.c |  0
 .../selftests/bpf/{ => kern_progs}/dev_cgroup.c    |  0
 .../bpf/{ => kern_progs}/get_cgroup_id_kern.c      |  0
 .../selftests/bpf/{ => kern_progs}/netcnt_prog.c   |  0
 .../bpf/{ => kern_progs}/sample_map_ret0.c         |  0
 .../selftests/bpf/{ => kern_progs}/sample_ret0.c   |  0
 .../selftests/bpf/{ => kern_progs}/sendmsg4_prog.c |  0
 .../selftests/bpf/{ => kern_progs}/sendmsg6_prog.c |  0
 .../bpf/{ => kern_progs}/socket_cookie_prog.c      |  0
 .../bpf/{ => kern_progs}/sockmap_parse_prog.c      |  0
 .../bpf/{ => kern_progs}/sockmap_tcp_msg_prog.c    |  0
 .../bpf/{ => kern_progs}/sockmap_verdict_prog.c    |  0
 .../bpf/{ => kern_progs}/test_adjust_tail.c        |  0
 .../bpf/{ => kern_progs}/test_btf_haskv.c          |  0
 .../selftests/bpf/{ => kern_progs}/test_btf_nokv.c |  0
 .../bpf/{ => kern_progs}/test_get_stack_rawtp.c    |  0
 .../selftests/bpf/{ => kern_progs}/test_l4lb.c     |  0
 .../bpf/{ => kern_progs}/test_l4lb_noinline.c      |  0
 .../bpf/{ => kern_progs}/test_lirc_mode2_kern.c    |  0
 .../bpf/{ => kern_progs}/test_lwt_seg6local.c      |  0
 .../bpf/{ => kern_progs}/test_map_in_map.c         |  0
 .../selftests/bpf/{ => kern_progs}/test_map_lock.c |  0
 .../selftests/bpf/{ => kern_progs}/test_obj_id.c   |  0
 .../bpf/{ => kern_progs}/test_pkt_access.c         |  0
 .../bpf/{ => kern_progs}/test_pkt_md_access.c      |  0
 .../bpf/{ => kern_progs}/test_queue_map.c          |  0
 .../{ => kern_progs}/test_select_reuseport_kern.c  |  0
 .../bpf/{ => kern_progs}/test_sk_lookup_kern.c     |  0
 .../bpf/{ => kern_progs}/test_skb_cgroup_id_kern.c |  0
 .../bpf/{ => kern_progs}/test_sockhash_kern.c      |  0
 .../bpf/{ => kern_progs}/test_sockmap_kern.c       |  0
 .../bpf/{ => kern_progs}/test_spin_lock.c          |  0
 .../bpf/{ => kern_progs}/test_stack_map.c          |  0
 .../{ => kern_progs}/test_stacktrace_build_id.c    |  0
 .../bpf/{ => kern_progs}/test_stacktrace_map.c     |  0
 .../bpf/{ => kern_progs}/test_tcp_estats.c         |  0
 .../bpf/{ => kern_progs}/test_tcpbpf_kern.c        |  0
 .../bpf/{ => kern_progs}/test_tcpnotify_kern.c     |  0
 .../bpf/{ => kern_progs}/test_tracepoint.c         |  0
 .../bpf/{ => kern_progs}/test_tunnel_kern.c        |  0
 .../selftests/bpf/{ => kern_progs}/test_xdp.c      |  0
 .../selftests/bpf/{ => kern_progs}/test_xdp_meta.c |  0
 .../bpf/{ => kern_progs}/test_xdp_noinline.c       |  0
 .../bpf/{ => kern_progs}/test_xdp_redirect.c       |  0
 .../selftests/bpf/{ => kern_progs}/test_xdp_vlan.c |  0
 .../selftests/bpf/{ => kern_progs}/xdp_dummy.c     |  0
 50 files changed, 8 insertions(+), 28 deletions(-)
 rename tools/testing/selftests/bpf/{ => kern_progs}/bpf_flow.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/connect4_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/connect6_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/dev_cgroup.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/get_cgroup_id_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/netcnt_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sample_map_ret0.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sample_ret0.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sendmsg4_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sendmsg6_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/socket_cookie_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sockmap_parse_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sockmap_tcp_msg_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sockmap_verdict_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_adjust_tail.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_btf_haskv.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_btf_nokv.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_get_stack_rawtp.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_l4lb.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_l4lb_noinline.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_lirc_mode2_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_lwt_seg6local.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_map_in_map.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_map_lock.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_obj_id.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_pkt_access.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_pkt_md_access.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_queue_map.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_select_reuseport_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_sk_lookup_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_skb_cgroup_id_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_sockhash_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_sockmap_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_spin_lock.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_stack_map.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_stacktrace_build_id.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_stacktrace_map.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_tcp_estats.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_tcpbpf_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_tcpnotify_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_tracepoint.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_tunnel_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_xdp.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_xdp_meta.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_xdp_noinline.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_xdp_redirect.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_xdp_vlan.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/xdp_dummy.c (100%)

-- 
2.7.4

[PATCH bpf-next 1/3] selftests: bpf: add "alu32" to .gitignore

[Jiong Wang]

"alu32" is a build dir and contains various files for BPF sub-register
code-gen testing.

This patch tells git to ignore it.

Suggested-by: Yonghong Song <yhs@fb.com>
Reviewed-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: Jiong Wang <jiong.wang@netronome.com>
---
 tools/testing/selftests/bpf/.gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index dd093bd..e47168d 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -29,3 +29,4 @@ test_netcnt
 test_section_names
 test_tcpnotify_user
 test_libbpf
+alu32
-- 
2.7.4

[PATCH bpf-next 2/3] selftests: bpf: extend sub-register mode compilation to all bpf object files

[Jiong Wang]

At the moment, we only do extra sub-register mode compilation on bpf object
files used by "test_progs". These object files are really loaded and
executed.

This patch further extends sub-register mode compilation to all bpf object
files, even those without corresponding runtime tests. Because this could
help testing LLVM sub-register code-gen, kernel bpf selftest has much more
C testcases with reasonable size and complexity compared with LLVM
testsuite which only contains unit tests.

There were some file duplication inside BPF_OBJ_FILES_DUAL_COMPILE which
is removed now.

Reviewed-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: Jiong Wang <jiong.wang@netronome.com>
---
 tools/testing/selftests/bpf/Makefile | 21 ++++++++-------------
 1 file changed, 8 insertions(+), 13 deletions(-)

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 383d2ff..70b2570 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -35,20 +35,15 @@ BPF_OBJ_FILES = \
 	sendmsg4_prog.o sendmsg6_prog.o test_lirc_mode2_kern.o \
 	get_cgroup_id_kern.o socket_cookie_prog.o test_select_reuseport_kern.o \
 	test_skb_cgroup_id_kern.o bpf_flow.o netcnt_prog.o test_xdp_vlan.o \
-	xdp_dummy.o test_map_in_map.o test_spin_lock.o test_map_lock.o
-
-# Objects are built with default compilation flags and with sub-register
-# code-gen enabled.
-BPF_OBJ_FILES_DUAL_COMPILE = \
-	test_pkt_access.o test_pkt_access.o test_xdp.o test_adjust_tail.o \
-	test_l4lb.o test_l4lb_noinline.o test_xdp_noinline.o test_tcp_estats.o \
+	xdp_dummy.o test_map_in_map.o test_spin_lock.o test_map_lock.o \
+	test_pkt_access.o test_xdp.o test_adjust_tail.o test_l4lb.o \
+	test_l4lb_noinline.o test_xdp_noinline.o test_tcp_estats.o \
 	test_obj_id.o test_pkt_md_access.o test_tracepoint.o \
-	test_stacktrace_map.o test_stacktrace_map.o test_stacktrace_build_id.o \
-	test_stacktrace_build_id.o test_get_stack_rawtp.o \
-	test_get_stack_rawtp.o test_tracepoint.o test_sk_lookup_kern.o \
-	test_queue_map.o test_stack_map.o
+	test_stacktrace_map.o test_stacktrace_build_id.o \
+	test_get_stack_rawtp.o test_sk_lookup_kern.o test_queue_map.o \
+	test_stack_map.o
 
-TEST_GEN_FILES = $(BPF_OBJ_FILES) $(BPF_OBJ_FILES_DUAL_COMPILE)
+TEST_GEN_FILES = $(BPF_OBJ_FILES)
 
 # Also test sub-register code-gen if LLVM + kernel both has eBPF v3 processor
 # support which is the first version to contain both ALU32 and JMP32
@@ -58,7 +53,7 @@ SUBREG_CODEGEN := $(shell echo "int cal(int a) { return a > 0; }" | \
 			$(LLC) -mattr=+alu32 -mcpu=probe 2>&1 | \
 			grep 'if w')
 ifneq ($(SUBREG_CODEGEN),)
-TEST_GEN_FILES += $(patsubst %.o,alu32/%.o, $(BPF_OBJ_FILES_DUAL_COMPILE))
+TEST_GEN_FILES += $(patsubst %.o,alu32/%.o, $(BPF_OBJ_FILES))
 endif
 
 # Order correspond to 'make run_tests' order
-- 
2.7.4

[PATCH bpf-next 3/3] selftests: bpf: centre kernel bpf objects under new subdir "kern_progs"

[Jiong Wang]

At the moment, all kernel bpf objects are listed under BPF_OBJ_FILES.
Listing them manually sometimes causing patch conflict when people are
adding new testcases simultaneously.

It is better to centre all the related source files under a subdir
"kern_progs", then auto-generate the object file list.

Suggested-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: Jiong Wang <jiong.wang@netronome.com>
---
 tools/testing/selftests/bpf/Makefile               | 26 +++++-----------------
 .../selftests/bpf/{ => kern_progs}/bpf_flow.c      |  0
 .../selftests/bpf/{ => kern_progs}/connect4_prog.c |  0
 .../selftests/bpf/{ => kern_progs}/connect6_prog.c |  0
 .../selftests/bpf/{ => kern_progs}/dev_cgroup.c    |  0
 .../bpf/{ => kern_progs}/get_cgroup_id_kern.c      |  0
 .../selftests/bpf/{ => kern_progs}/netcnt_prog.c   |  0
 .../bpf/{ => kern_progs}/sample_map_ret0.c         |  0
 .../selftests/bpf/{ => kern_progs}/sample_ret0.c   |  0
 .../selftests/bpf/{ => kern_progs}/sendmsg4_prog.c |  0
 .../selftests/bpf/{ => kern_progs}/sendmsg6_prog.c |  0
 .../bpf/{ => kern_progs}/socket_cookie_prog.c      |  0
 .../bpf/{ => kern_progs}/sockmap_parse_prog.c      |  0
 .../bpf/{ => kern_progs}/sockmap_tcp_msg_prog.c    |  0
 .../bpf/{ => kern_progs}/sockmap_verdict_prog.c    |  0
 .../bpf/{ => kern_progs}/test_adjust_tail.c        |  0
 .../bpf/{ => kern_progs}/test_btf_haskv.c          |  0
 .../selftests/bpf/{ => kern_progs}/test_btf_nokv.c |  0
 .../bpf/{ => kern_progs}/test_get_stack_rawtp.c    |  0
 .../selftests/bpf/{ => kern_progs}/test_l4lb.c     |  0
 .../bpf/{ => kern_progs}/test_l4lb_noinline.c      |  0
 .../bpf/{ => kern_progs}/test_lirc_mode2_kern.c    |  0
 .../bpf/{ => kern_progs}/test_lwt_seg6local.c      |  0
 .../bpf/{ => kern_progs}/test_map_in_map.c         |  0
 .../selftests/bpf/{ => kern_progs}/test_map_lock.c |  0
 .../selftests/bpf/{ => kern_progs}/test_obj_id.c   |  0
 .../bpf/{ => kern_progs}/test_pkt_access.c         |  0
 .../bpf/{ => kern_progs}/test_pkt_md_access.c      |  0
 .../bpf/{ => kern_progs}/test_queue_map.c          |  0
 .../{ => kern_progs}/test_select_reuseport_kern.c  |  0
 .../bpf/{ => kern_progs}/test_sk_lookup_kern.c     |  0
 .../bpf/{ => kern_progs}/test_skb_cgroup_id_kern.c |  0
 .../bpf/{ => kern_progs}/test_sockhash_kern.c      |  0
 .../bpf/{ => kern_progs}/test_sockmap_kern.c       |  0
 .../bpf/{ => kern_progs}/test_spin_lock.c          |  0
 .../bpf/{ => kern_progs}/test_stack_map.c          |  0
 .../{ => kern_progs}/test_stacktrace_build_id.c    |  0
 .../bpf/{ => kern_progs}/test_stacktrace_map.c     |  0
 .../bpf/{ => kern_progs}/test_tcp_estats.c         |  0
 .../bpf/{ => kern_progs}/test_tcpbpf_kern.c        |  0
 .../bpf/{ => kern_progs}/test_tcpnotify_kern.c     |  0
 .../bpf/{ => kern_progs}/test_tracepoint.c         |  0
 .../bpf/{ => kern_progs}/test_tunnel_kern.c        |  0
 .../selftests/bpf/{ => kern_progs}/test_xdp.c      |  0
 .../selftests/bpf/{ => kern_progs}/test_xdp_meta.c |  0
 .../bpf/{ => kern_progs}/test_xdp_noinline.c       |  0
 .../bpf/{ => kern_progs}/test_xdp_redirect.c       |  0
 .../selftests/bpf/{ => kern_progs}/test_xdp_vlan.c |  0
 .../selftests/bpf/{ => kern_progs}/xdp_dummy.c     |  0
 49 files changed, 5 insertions(+), 21 deletions(-)
 rename tools/testing/selftests/bpf/{ => kern_progs}/bpf_flow.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/connect4_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/connect6_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/dev_cgroup.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/get_cgroup_id_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/netcnt_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sample_map_ret0.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sample_ret0.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sendmsg4_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sendmsg6_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/socket_cookie_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sockmap_parse_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sockmap_tcp_msg_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/sockmap_verdict_prog.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_adjust_tail.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_btf_haskv.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_btf_nokv.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_get_stack_rawtp.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_l4lb.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_l4lb_noinline.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_lirc_mode2_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_lwt_seg6local.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_map_in_map.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_map_lock.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_obj_id.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_pkt_access.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_pkt_md_access.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_queue_map.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_select_reuseport_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_sk_lookup_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_skb_cgroup_id_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_sockhash_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_sockmap_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_spin_lock.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_stack_map.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_stacktrace_build_id.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_stacktrace_map.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_tcp_estats.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_tcpbpf_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_tcpnotify_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_tracepoint.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_tunnel_kern.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_xdp.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_xdp_meta.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_xdp_noinline.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_xdp_redirect.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/test_xdp_vlan.c (100%)
 rename tools/testing/selftests/bpf/{ => kern_progs}/xdp_dummy.c (100%)

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 70b2570..2965855 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -25,24 +25,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test
 	test_socket_cookie test_cgroup_storage test_select_reuseport test_section_names \
 	test_netcnt test_tcpnotify_user
 
-BPF_OBJ_FILES = \
-	test_xdp_redirect.o test_xdp_meta.o sockmap_parse_prog.o \
-	sockmap_verdict_prog.o dev_cgroup.o sample_ret0.o \
-	test_tcpnotify_kern.o sample_map_ret0.o test_tcpbpf_kern.o \
-	sockmap_tcp_msg_prog.o connect4_prog.o connect6_prog.o \
-	test_btf_haskv.o test_btf_nokv.o test_sockmap_kern.o \
-	test_tunnel_kern.o test_sockhash_kern.o test_lwt_seg6local.o \
-	sendmsg4_prog.o sendmsg6_prog.o test_lirc_mode2_kern.o \
-	get_cgroup_id_kern.o socket_cookie_prog.o test_select_reuseport_kern.o \
-	test_skb_cgroup_id_kern.o bpf_flow.o netcnt_prog.o test_xdp_vlan.o \
-	xdp_dummy.o test_map_in_map.o test_spin_lock.o test_map_lock.o \
-	test_pkt_access.o test_xdp.o test_adjust_tail.o test_l4lb.o \
-	test_l4lb_noinline.o test_xdp_noinline.o test_tcp_estats.o \
-	test_obj_id.o test_pkt_md_access.o test_tracepoint.o \
-	test_stacktrace_map.o test_stacktrace_build_id.o \
-	test_get_stack_rawtp.o test_sk_lookup_kern.o test_queue_map.o \
-	test_stack_map.o
-
+BPF_OBJ_FILES = $(patsubst %.c,%.o, $(notdir $(wildcard kern_progs/*.c)))
 TEST_GEN_FILES = $(BPF_OBJ_FILES)
 
 # Also test sub-register code-gen if LLVM + kernel both has eBPF v3 processor
@@ -183,7 +166,8 @@ $(ALU32_BUILD_DIR)/test_progs_32: test_progs.c $(ALU32_BUILD_DIR) \
 	$(CC) $(CFLAGS) -o $(ALU32_BUILD_DIR)/test_progs_32 $< \
 		trace_helpers.c $(OUTPUT)/libbpf.a $(LDLIBS)
 
-$(ALU32_BUILD_DIR)/%.o: %.c $(ALU32_BUILD_DIR) $(ALU32_BUILD_DIR)/test_progs_32
+$(ALU32_BUILD_DIR)/%.o: kern_progs/%.c $(ALU32_BUILD_DIR) \
+					$(ALU32_BUILD_DIR)/test_progs_32
 	$(CLANG) $(CLANG_FLAGS) \
 		 -O2 -target bpf -emit-llvm -c $< -o - |      \
 	$(LLC) -march=bpf -mattr=+alu32 -mcpu=$(CPU) $(LLC_FLAGS) \
@@ -195,7 +179,7 @@ endif
 
 # Have one program compiled without "-target bpf" to test whether libbpf loads
 # it successfully
-$(OUTPUT)/test_xdp.o: test_xdp.c
+$(OUTPUT)/test_xdp.o: kern_progs/test_xdp.c
 	$(CLANG) $(CLANG_FLAGS) \
 		-O2 -emit-llvm -c $< -o - | \
 	$(LLC) -march=bpf -mcpu=$(CPU) $(LLC_FLAGS) -filetype=obj -o $@
@@ -203,7 +187,7 @@ ifeq ($(DWARF2BTF),y)
 	$(BTF_PAHOLE) -J $@
 endif
 
-$(OUTPUT)/%.o: %.c
+$(OUTPUT)/%.o: kern_progs/%.c
 	$(CLANG) $(CLANG_FLAGS) \
 		 -O2 -target bpf -emit-llvm -c $< -o - |      \
 	$(LLC) -march=bpf -mcpu=$(CPU) $(LLC_FLAGS) -filetype=obj -o $@
diff --git a/tools/testing/selftests/bpf/bpf_flow.c b/tools/testing/selftests/bpf/kern_progs/bpf_flow.c
similarity index 100%
rename from tools/testing/selftests/bpf/bpf_flow.c
rename to tools/testing/selftests/bpf/kern_progs/bpf_flow.c
diff --git a/tools/testing/selftests/bpf/connect4_prog.c b/tools/testing/selftests/bpf/kern_progs/connect4_prog.c
similarity index 100%
rename from tools/testing/selftests/bpf/connect4_prog.c
rename to tools/testing/selftests/bpf/kern_progs/connect4_prog.c
diff --git a/tools/testing/selftests/bpf/connect6_prog.c b/tools/testing/selftests/bpf/kern_progs/connect6_prog.c
similarity index 100%
rename from tools/testing/selftests/bpf/connect6_prog.c
rename to tools/testing/selftests/bpf/kern_progs/connect6_prog.c
diff --git a/tools/testing/selftests/bpf/dev_cgroup.c b/tools/testing/selftests/bpf/kern_progs/dev_cgroup.c
similarity index 100%
rename from tools/testing/selftests/bpf/dev_cgroup.c
rename to tools/testing/selftests/bpf/kern_progs/dev_cgroup.c
diff --git a/tools/testing/selftests/bpf/get_cgroup_id_kern.c b/tools/testing/selftests/bpf/kern_progs/get_cgroup_id_kern.c
similarity index 100%
rename from tools/testing/selftests/bpf/get_cgroup_id_kern.c
rename to tools/testing/selftests/bpf/kern_progs/get_cgroup_id_kern.c
diff --git a/tools/testing/selftests/bpf/netcnt_prog.c b/tools/testing/selftests/bpf/kern_progs/netcnt_prog.c
similarity index 100%
rename from tools/testing/selftests/bpf/netcnt_prog.c
rename to tools/testing/selftests/bpf/kern_progs/netcnt_prog.c
diff --git a/tools/testing/selftests/bpf/sample_map_ret0.c b/tools/testing/selftests/bpf/kern_progs/sample_map_ret0.c
similarity index 100%
rename from tools/testing/selftests/bpf/sample_map_ret0.c
rename to tools/testing/selftests/bpf/kern_progs/sample_map_ret0.c
diff --git a/tools/testing/selftests/bpf/sample_ret0.c b/tools/testing/selftests/bpf/kern_progs/sample_ret0.c
similarity index 100%
rename from tools/testing/selftests/bpf/sample_ret0.c
rename to tools/testing/selftests/bpf/kern_progs/sample_ret0.c
diff --git a/tools/testing/selftests/bpf/sendmsg4_prog.c b/tools/testing/selftests/bpf/kern_progs/sendmsg4_prog.c
similarity index 100%
rename from tools/testing/selftests/bpf/sendmsg4_prog.c
rename to tools/testing/selftests/bpf/kern_progs/sendmsg4_prog.c
diff --git a/tools/testing/selftests/bpf/sendmsg6_prog.c b/tools/testing/selftests/bpf/kern_progs/sendmsg6_prog.c
similarity index 100%
rename from tools/testing/selftests/bpf/sendmsg6_prog.c
rename to tools/testing/selftests/bpf/kern_progs/sendmsg6_prog.c
diff --git a/tools/testing/selftests/bpf/socket_cookie_prog.c b/tools/testing/selftests/bpf/kern_progs/socket_cookie_prog.c
similarity index 100%
rename from tools/testing/selftests/bpf/socket_cookie_prog.c
rename to tools/testing/selftests/bpf/kern_progs/socket_cookie_prog.c
diff --git a/tools/testing/selftests/bpf/sockmap_parse_prog.c b/tools/testing/selftests/bpf/kern_progs/sockmap_parse_prog.c
similarity index 100%
rename from tools/testing/selftests/bpf/sockmap_parse_prog.c
rename to tools/testing/selftests/bpf/kern_progs/sockmap_parse_prog.c
diff --git a/tools/testing/selftests/bpf/sockmap_tcp_msg_prog.c b/tools/testing/selftests/bpf/kern_progs/sockmap_tcp_msg_prog.c
similarity index 100%
rename from tools/testing/selftests/bpf/sockmap_tcp_msg_prog.c
rename to tools/testing/selftests/bpf/kern_progs/sockmap_tcp_msg_prog.c
diff --git a/tools/testing/selftests/bpf/sockmap_verdict_prog.c b/tools/testing/selftests/bpf/kern_progs/sockmap_verdict_prog.c
similarity index 100%
rename from tools/testing/selftests/bpf/sockmap_verdict_prog.c
rename to tools/testing/selftests/bpf/kern_progs/sockmap_verdict_prog.c
diff --git a/tools/testing/selftests/bpf/test_adjust_tail.c b/tools/testing/selftests/bpf/kern_progs/test_adjust_tail.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_adjust_tail.c
rename to tools/testing/selftests/bpf/kern_progs/test_adjust_tail.c
diff --git a/tools/testing/selftests/bpf/test_btf_haskv.c b/tools/testing/selftests/bpf/kern_progs/test_btf_haskv.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_btf_haskv.c
rename to tools/testing/selftests/bpf/kern_progs/test_btf_haskv.c
diff --git a/tools/testing/selftests/bpf/test_btf_nokv.c b/tools/testing/selftests/bpf/kern_progs/test_btf_nokv.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_btf_nokv.c
rename to tools/testing/selftests/bpf/kern_progs/test_btf_nokv.c
diff --git a/tools/testing/selftests/bpf/test_get_stack_rawtp.c b/tools/testing/selftests/bpf/kern_progs/test_get_stack_rawtp.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_get_stack_rawtp.c
rename to tools/testing/selftests/bpf/kern_progs/test_get_stack_rawtp.c
diff --git a/tools/testing/selftests/bpf/test_l4lb.c b/tools/testing/selftests/bpf/kern_progs/test_l4lb.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_l4lb.c
rename to tools/testing/selftests/bpf/kern_progs/test_l4lb.c
diff --git a/tools/testing/selftests/bpf/test_l4lb_noinline.c b/tools/testing/selftests/bpf/kern_progs/test_l4lb_noinline.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_l4lb_noinline.c
rename to tools/testing/selftests/bpf/kern_progs/test_l4lb_noinline.c
diff --git a/tools/testing/selftests/bpf/test_lirc_mode2_kern.c b/tools/testing/selftests/bpf/kern_progs/test_lirc_mode2_kern.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_lirc_mode2_kern.c
rename to tools/testing/selftests/bpf/kern_progs/test_lirc_mode2_kern.c
diff --git a/tools/testing/selftests/bpf/test_lwt_seg6local.c b/tools/testing/selftests/bpf/kern_progs/test_lwt_seg6local.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_lwt_seg6local.c
rename to tools/testing/selftests/bpf/kern_progs/test_lwt_seg6local.c
diff --git a/tools/testing/selftests/bpf/test_map_in_map.c b/tools/testing/selftests/bpf/kern_progs/test_map_in_map.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_map_in_map.c
rename to tools/testing/selftests/bpf/kern_progs/test_map_in_map.c
diff --git a/tools/testing/selftests/bpf/test_map_lock.c b/tools/testing/selftests/bpf/kern_progs/test_map_lock.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_map_lock.c
rename to tools/testing/selftests/bpf/kern_progs/test_map_lock.c
diff --git a/tools/testing/selftests/bpf/test_obj_id.c b/tools/testing/selftests/bpf/kern_progs/test_obj_id.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_obj_id.c
rename to tools/testing/selftests/bpf/kern_progs/test_obj_id.c
diff --git a/tools/testing/selftests/bpf/test_pkt_access.c b/tools/testing/selftests/bpf/kern_progs/test_pkt_access.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_pkt_access.c
rename to tools/testing/selftests/bpf/kern_progs/test_pkt_access.c
diff --git a/tools/testing/selftests/bpf/test_pkt_md_access.c b/tools/testing/selftests/bpf/kern_progs/test_pkt_md_access.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_pkt_md_access.c
rename to tools/testing/selftests/bpf/kern_progs/test_pkt_md_access.c
diff --git a/tools/testing/selftests/bpf/test_queue_map.c b/tools/testing/selftests/bpf/kern_progs/test_queue_map.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_queue_map.c
rename to tools/testing/selftests/bpf/kern_progs/test_queue_map.c
diff --git a/tools/testing/selftests/bpf/test_select_reuseport_kern.c b/tools/testing/selftests/bpf/kern_progs/test_select_reuseport_kern.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_select_reuseport_kern.c
rename to tools/testing/selftests/bpf/kern_progs/test_select_reuseport_kern.c
diff --git a/tools/testing/selftests/bpf/test_sk_lookup_kern.c b/tools/testing/selftests/bpf/kern_progs/test_sk_lookup_kern.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_sk_lookup_kern.c
rename to tools/testing/selftests/bpf/kern_progs/test_sk_lookup_kern.c
diff --git a/tools/testing/selftests/bpf/test_skb_cgroup_id_kern.c b/tools/testing/selftests/bpf/kern_progs/test_skb_cgroup_id_kern.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_skb_cgroup_id_kern.c
rename to tools/testing/selftests/bpf/kern_progs/test_skb_cgroup_id_kern.c
diff --git a/tools/testing/selftests/bpf/test_sockhash_kern.c b/tools/testing/selftests/bpf/kern_progs/test_sockhash_kern.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_sockhash_kern.c
rename to tools/testing/selftests/bpf/kern_progs/test_sockhash_kern.c
diff --git a/tools/testing/selftests/bpf/test_sockmap_kern.c b/tools/testing/selftests/bpf/kern_progs/test_sockmap_kern.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_sockmap_kern.c
rename to tools/testing/selftests/bpf/kern_progs/test_sockmap_kern.c
diff --git a/tools/testing/selftests/bpf/test_spin_lock.c b/tools/testing/selftests/bpf/kern_progs/test_spin_lock.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_spin_lock.c
rename to tools/testing/selftests/bpf/kern_progs/test_spin_lock.c
diff --git a/tools/testing/selftests/bpf/test_stack_map.c b/tools/testing/selftests/bpf/kern_progs/test_stack_map.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_stack_map.c
rename to tools/testing/selftests/bpf/kern_progs/test_stack_map.c
diff --git a/tools/testing/selftests/bpf/test_stacktrace_build_id.c b/tools/testing/selftests/bpf/kern_progs/test_stacktrace_build_id.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_stacktrace_build_id.c
rename to tools/testing/selftests/bpf/kern_progs/test_stacktrace_build_id.c
diff --git a/tools/testing/selftests/bpf/test_stacktrace_map.c b/tools/testing/selftests/bpf/kern_progs/test_stacktrace_map.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_stacktrace_map.c
rename to tools/testing/selftests/bpf/kern_progs/test_stacktrace_map.c
diff --git a/tools/testing/selftests/bpf/test_tcp_estats.c b/tools/testing/selftests/bpf/kern_progs/test_tcp_estats.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_tcp_estats.c
rename to tools/testing/selftests/bpf/kern_progs/test_tcp_estats.c
diff --git a/tools/testing/selftests/bpf/test_tcpbpf_kern.c b/tools/testing/selftests/bpf/kern_progs/test_tcpbpf_kern.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_tcpbpf_kern.c
rename to tools/testing/selftests/bpf/kern_progs/test_tcpbpf_kern.c
diff --git a/tools/testing/selftests/bpf/test_tcpnotify_kern.c b/tools/testing/selftests/bpf/kern_progs/test_tcpnotify_kern.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_tcpnotify_kern.c
rename to tools/testing/selftests/bpf/kern_progs/test_tcpnotify_kern.c
diff --git a/tools/testing/selftests/bpf/test_tracepoint.c b/tools/testing/selftests/bpf/kern_progs/test_tracepoint.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_tracepoint.c
rename to tools/testing/selftests/bpf/kern_progs/test_tracepoint.c
diff --git a/tools/testing/selftests/bpf/test_tunnel_kern.c b/tools/testing/selftests/bpf/kern_progs/test_tunnel_kern.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_tunnel_kern.c
rename to tools/testing/selftests/bpf/kern_progs/test_tunnel_kern.c
diff --git a/tools/testing/selftests/bpf/test_xdp.c b/tools/testing/selftests/bpf/kern_progs/test_xdp.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_xdp.c
rename to tools/testing/selftests/bpf/kern_progs/test_xdp.c
diff --git a/tools/testing/selftests/bpf/test_xdp_meta.c b/tools/testing/selftests/bpf/kern_progs/test_xdp_meta.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_xdp_meta.c
rename to tools/testing/selftests/bpf/kern_progs/test_xdp_meta.c
diff --git a/tools/testing/selftests/bpf/test_xdp_noinline.c b/tools/testing/selftests/bpf/kern_progs/test_xdp_noinline.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_xdp_noinline.c
rename to tools/testing/selftests/bpf/kern_progs/test_xdp_noinline.c
diff --git a/tools/testing/selftests/bpf/test_xdp_redirect.c b/tools/testing/selftests/bpf/kern_progs/test_xdp_redirect.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_xdp_redirect.c
rename to tools/testing/selftests/bpf/kern_progs/test_xdp_redirect.c
diff --git a/tools/testing/selftests/bpf/test_xdp_vlan.c b/tools/testing/selftests/bpf/kern_progs/test_xdp_vlan.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_xdp_vlan.c
rename to tools/testing/selftests/bpf/kern_progs/test_xdp_vlan.c
diff --git a/tools/testing/selftests/bpf/xdp_dummy.c b/tools/testing/selftests/bpf/kern_progs/xdp_dummy.c
similarity index 100%
rename from tools/testing/selftests/bpf/xdp_dummy.c
rename to tools/testing/selftests/bpf/kern_progs/xdp_dummy.c
-- 
2.7.4

Re: [PATCH bpf-next 2/3] selftests: bpf: extend sub-register mode compilation to all bpf object files

[Alexei Starovoitov]

On Fri, Feb 08, 2019 at 05:41:20PM +0000, Jiong Wang wrote:
> At the moment, we only do extra sub-register mode compilation on bpf object
> files used by "test_progs". These object files are really loaded and
> executed.
> 
> This patch further extends sub-register mode compilation to all bpf object
> files, even those without corresponding runtime tests. Because this could
> help testing LLVM sub-register code-gen, kernel bpf selftest has much more
> C testcases with reasonable size and complexity compared with LLVM
> testsuite which only contains unit tests.
> 
> There were some file duplication inside BPF_OBJ_FILES_DUAL_COMPILE which
> is removed now.
> 
> Reviewed-by: Jakub Kicinski <jakub.kicinski@netronome.com>
> Signed-off-by: Jiong Wang <jiong.wang@netronome.com>
> ---
>  tools/testing/selftests/bpf/Makefile | 21 ++++++++-------------
>  1 file changed, 8 insertions(+), 13 deletions(-)
> 
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 383d2ff..70b2570 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -35,20 +35,15 @@ BPF_OBJ_FILES = \
>  	sendmsg4_prog.o sendmsg6_prog.o test_lirc_mode2_kern.o \
>  	get_cgroup_id_kern.o socket_cookie_prog.o test_select_reuseport_kern.o \
>  	test_skb_cgroup_id_kern.o bpf_flow.o netcnt_prog.o test_xdp_vlan.o \
> -	xdp_dummy.o test_map_in_map.o test_spin_lock.o test_map_lock.o
> -
> -# Objects are built with default compilation flags and with sub-register
> -# code-gen enabled.
> -BPF_OBJ_FILES_DUAL_COMPILE = \
> -	test_pkt_access.o test_pkt_access.o test_xdp.o test_adjust_tail.o \
> -	test_l4lb.o test_l4lb_noinline.o test_xdp_noinline.o test_tcp_estats.o \
> +	xdp_dummy.o test_map_in_map.o test_spin_lock.o test_map_lock.o \
> +	test_pkt_access.o test_xdp.o test_adjust_tail.o test_l4lb.o \
> +	test_l4lb_noinline.o test_xdp_noinline.o test_tcp_estats.o \
>  	test_obj_id.o test_pkt_md_access.o test_tracepoint.o \
> -	test_stacktrace_map.o test_stacktrace_map.o test_stacktrace_build_id.o \
> -	test_stacktrace_build_id.o test_get_stack_rawtp.o \
> -	test_get_stack_rawtp.o test_tracepoint.o test_sk_lookup_kern.o \
> -	test_queue_map.o test_stack_map.o
> +	test_stacktrace_map.o test_stacktrace_build_id.o \
> +	test_get_stack_rawtp.o test_sk_lookup_kern.o test_queue_map.o \
> +	test_stack_map.o
>  
> -TEST_GEN_FILES = $(BPF_OBJ_FILES) $(BPF_OBJ_FILES_DUAL_COMPILE)
> +TEST_GEN_FILES = $(BPF_OBJ_FILES)
>  
>  # Also test sub-register code-gen if LLVM + kernel both has eBPF v3 processor
>  # support which is the first version to contain both ALU32 and JMP32
> @@ -58,7 +53,7 @@ SUBREG_CODEGEN := $(shell echo "int cal(int a) { return a > 0; }" | \
>  			$(LLC) -mattr=+alu32 -mcpu=probe 2>&1 | \
>  			grep 'if w')

build and test servers can be different.
Would it make sense to use -mcpu=v3 instead of -mcpu=probe ?

Also while testing test_progs_32 fails like this:
libbpf: failed to open ./bpf_flow.o: No such file or directory
libbpf: failed to open ./test_spin_lock.o: No such file or directory
test_spin_lock:bpf_prog_load errno 2

Do you see the same ?

Re: [PATCH bpf-next 3/3] selftests: bpf: centre kernel bpf objects under new subdir "kern_progs"

[Alexei Starovoitov]

On Fri, Feb 08, 2019 at 05:41:21PM +0000, Jiong Wang wrote:
> At the moment, all kernel bpf objects are listed under BPF_OBJ_FILES.
> Listing them manually sometimes causing patch conflict when people are
> adding new testcases simultaneously.
> 
> It is better to centre all the related source files under a subdir
> "kern_progs", then auto-generate the object file list.
> 
> Suggested-by: Alexei Starovoitov <ast@kernel.org>
> Reviewed-by: Jakub Kicinski <jakub.kicinski@netronome.com>
> Signed-off-by: Jiong Wang <jiong.wang@netronome.com>
> ---
>  tools/testing/selftests/bpf/Makefile               | 26 +++++-----------------
>  .../selftests/bpf/{ => kern_progs}/bpf_flow.c      |  0
>  .../selftests/bpf/{ => kern_progs}/connect4_prog.c |  0
>  .../selftests/bpf/{ => kern_progs}/connect6_prog.c |  0
>  .../selftests/bpf/{ => kern_progs}/dev_cgroup.c    |  0

Thanks a lot for the patch.
A tiny bit of bikeshedding...
'kern_progs' feels a bit too long and awkward to type.
May be just 'progs' ?

Re: [PATCH bpf-next 2/3] selftests: bpf: extend sub-register mode compilation to all bpf object files

[Jiong Wang]

> On 11 Feb 2019, at 04:04, Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote:
> 
> On Fri, Feb 08, 2019 at 05:41:20PM +0000, Jiong Wang wrote:
>> At the moment, we only do extra sub-register mode compilation on bpf object
>> files used by "test_progs". These object files are really loaded and
>> executed.
>> 
>> This patch further extends sub-register mode compilation to all bpf object
>> files, even those without corresponding runtime tests. Because this could
>> help testing LLVM sub-register code-gen, kernel bpf selftest has much more
>> C testcases with reasonable size and complexity compared with LLVM
>> testsuite which only contains unit tests.
>> 
>> There were some file duplication inside BPF_OBJ_FILES_DUAL_COMPILE which
>> is removed now.
>> 
>> Reviewed-by: Jakub Kicinski <jakub.kicinski@netronome.com>
>> Signed-off-by: Jiong Wang <jiong.wang@netronome.com>
>> ---
>> tools/testing/selftests/bpf/Makefile | 21 ++++++++-------------
>> 1 file changed, 8 insertions(+), 13 deletions(-)
>> 
>> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
>> index 383d2ff..70b2570 100644
>> --- a/tools/testing/selftests/bpf/Makefile
>> +++ b/tools/testing/selftests/bpf/Makefile
>> @@ -35,20 +35,15 @@ BPF_OBJ_FILES = \
>> 	sendmsg4_prog.o sendmsg6_prog.o test_lirc_mode2_kern.o \
>> 	get_cgroup_id_kern.o socket_cookie_prog.o test_select_reuseport_kern.o \
>> 	test_skb_cgroup_id_kern.o bpf_flow.o netcnt_prog.o test_xdp_vlan.o \
>> -	xdp_dummy.o test_map_in_map.o test_spin_lock.o test_map_lock.o
>> -
>> -# Objects are built with default compilation flags and with sub-register
>> -# code-gen enabled.
>> -BPF_OBJ_FILES_DUAL_COMPILE = \
>> -	test_pkt_access.o test_pkt_access.o test_xdp.o test_adjust_tail.o \
>> -	test_l4lb.o test_l4lb_noinline.o test_xdp_noinline.o test_tcp_estats.o \
>> +	xdp_dummy.o test_map_in_map.o test_spin_lock.o test_map_lock.o \
>> +	test_pkt_access.o test_xdp.o test_adjust_tail.o test_l4lb.o \
>> +	test_l4lb_noinline.o test_xdp_noinline.o test_tcp_estats.o \
>> 	test_obj_id.o test_pkt_md_access.o test_tracepoint.o \
>> -	test_stacktrace_map.o test_stacktrace_map.o test_stacktrace_build_id.o \
>> -	test_stacktrace_build_id.o test_get_stack_rawtp.o \
>> -	test_get_stack_rawtp.o test_tracepoint.o test_sk_lookup_kern.o \
>> -	test_queue_map.o test_stack_map.o
>> +	test_stacktrace_map.o test_stacktrace_build_id.o \
>> +	test_get_stack_rawtp.o test_sk_lookup_kern.o test_queue_map.o \
>> +	test_stack_map.o
>> 
>> -TEST_GEN_FILES = $(BPF_OBJ_FILES) $(BPF_OBJ_FILES_DUAL_COMPILE)
>> +TEST_GEN_FILES = $(BPF_OBJ_FILES)
>> 
>> # Also test sub-register code-gen if LLVM + kernel both has eBPF v3 processor
>> # support which is the first version to contain both ALU32 and JMP32
>> @@ -58,7 +53,7 @@ SUBREG_CODEGEN := $(shell echo "int cal(int a) { return a > 0; }" | \
>> 			$(LLC) -mattr=+alu32 -mcpu=probe 2>&1 | \
>> 			grep 'if w')
> 
> build and test servers can be different.
> Would it make sense to use -mcpu=v3 instead of -mcpu=probe ?

Have the same thoughts initially, but was worried the situation where doing native
bpf selftest, build/test on same machine and llvm is new but kernel is old.

I think remove the runtime kernel criteria make sense if for bpf selftest, separation
between build and test server are often, for example cross-compilation are used.

Will add a new patch to make this change.

> 
> Also while testing test_progs_32 fails like this:
> libbpf: failed to open ./bpf_flow.o: No such file or directory
> libbpf: failed to open ./test_spin_lock.o: No such file or directory
> test_spin_lock:bpf_prog_load errno 2
> 
> Do you see the same ?

Hmm, I haven’t see the same. This is expected to happen before this patch, but should
not after. Because bpf_flow and test_spin_lock were added after the initial JMP32 patch
set.

Noticed there is patch conflict between this patch and KaFai’s latest test_sock_fields
change. Perhaps this patch hasn’t been applied successfully when doing the test on your
env?

Regards,
Jiong

Re: [PATCH bpf-next 3/3] selftests: bpf: centre kernel bpf objects under new subdir "kern_progs"

[Jiong Wang]

> On 11 Feb 2019, at 04:06, Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote:
> 
> On Fri, Feb 08, 2019 at 05:41:21PM +0000, Jiong Wang wrote:
>> At the moment, all kernel bpf objects are listed under BPF_OBJ_FILES.
>> Listing them manually sometimes causing patch conflict when people are
>> adding new testcases simultaneously.
>> 
>> It is better to centre all the related source files under a subdir
>> "kern_progs", then auto-generate the object file list.
>> 
>> Suggested-by: Alexei Starovoitov <ast@kernel.org>
>> Reviewed-by: Jakub Kicinski <jakub.kicinski@netronome.com>
>> Signed-off-by: Jiong Wang <jiong.wang@netronome.com>
>> ---
>> tools/testing/selftests/bpf/Makefile               | 26 +++++-----------------
>> .../selftests/bpf/{ => kern_progs}/bpf_flow.c      |  0
>> .../selftests/bpf/{ => kern_progs}/connect4_prog.c |  0
>> .../selftests/bpf/{ => kern_progs}/connect6_prog.c |  0
>> .../selftests/bpf/{ => kern_progs}/dev_cgroup.c    |  0
> 
> Thanks a lot for the patch.
> A tiny bit of bikeshedding...
> 'kern_progs' feels a bit too long and awkward to type.
> May be just 'progs’ ?

Ack, will do the change in v2.

Regards,
Jiong