BPF Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH bpf-next 1/2] tools/bpftool: allow substituting custom vmlinux.h for the build
@ 2020-06-30  0:47 Andrii Nakryiko
  2020-06-30  0:47 ` [PATCH bpf-next 2/2] selftests/bpf: allow substituting custom vmlinux.h for selftests build Andrii Nakryiko
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Andrii Nakryiko @ 2020-06-30  0:47 UTC (permalink / raw)
  To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko

In some build contexts (e.g., Travis CI build for outdated kernel), vmlinux.h,
generated from available kernel, doesn't contain all the types necessary for
BPF program compilation. For such set up, the most maintainable way to deal
with this problem is to keep pre-generated (almost up-to-date) vmlinux.h
checked in and use it for compilation purposes. bpftool after that can deal
with kernel missing some of the features in runtime with no problems.

To that effect, allow to specify path to custom vmlinux.h to bpftool's
Makefile with VMLINUX_H variable.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
 tools/bpf/bpftool/Makefile | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index 8c6563e56ffc..273da1615503 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -122,20 +122,24 @@ BPFTOOL_BOOTSTRAP := $(if $(OUTPUT),$(OUTPUT)bpftool-bootstrap,./bpftool-bootstr
 BOOTSTRAP_OBJS = $(addprefix $(OUTPUT),main.o common.o json_writer.o gen.o btf.o)
 OBJS = $(patsubst %.c,$(OUTPUT)%.o,$(SRCS)) $(OUTPUT)disasm.o
 
-VMLINUX_BTF_PATHS := $(if $(O),$(O)/vmlinux)				\
+VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux)				\
 		     $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux)	\
 		     ../../../vmlinux					\
 		     /sys/kernel/btf/vmlinux				\
 		     /boot/vmlinux-$(shell uname -r)
-VMLINUX_BTF := $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS))))
+VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS))))
 
-ifneq ($(VMLINUX_BTF),)
+ifneq ($(VMLINUX_BTF)$(VMLINUX_H),)
 ifeq ($(feature-clang-bpf-co-re),1)
 
 BUILD_BPF_SKELS := 1
 
 $(OUTPUT)vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL_BOOTSTRAP)
+ifeq ($(VMLINUX_H),)
 	$(QUIET_GEN)$(BPFTOOL_BOOTSTRAP) btf dump file $< format c > $@
+else
+	$(Q)cp "$(VMLINUX_H)" $@
+endif
 
 $(OUTPUT)%.bpf.o: skeleton/%.bpf.c $(OUTPUT)vmlinux.h $(LIBBPF)
 	$(QUIET_CLANG)$(CLANG) \
-- 
2.24.1


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH bpf-next 2/2] selftests/bpf: allow substituting custom vmlinux.h for selftests build
  2020-06-30  0:47 [PATCH bpf-next 1/2] tools/bpftool: allow substituting custom vmlinux.h for the build Andrii Nakryiko
@ 2020-06-30  0:47 ` Andrii Nakryiko
  2020-06-30  4:51   ` Yonghong Song
  2020-06-30  3:12 ` [PATCH bpf-next 1/2] tools/bpftool: allow substituting custom vmlinux.h for the build Yonghong Song
  2020-06-30 14:12 ` Daniel Borkmann
  2 siblings, 1 reply; 5+ messages in thread
From: Andrii Nakryiko @ 2020-06-30  0:47 UTC (permalink / raw)
  To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko

Similarly to bpftool Makefile, allow to specify custom location of vmlinux.h
to be used during the build. This allows simpler testing setups with
checked-in pre-generated vmlinux.h.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
 tools/testing/selftests/bpf/Makefile | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 22aaec74ea0a..1f9c696b3edf 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -134,12 +134,12 @@ $(OUTPUT)/test_stub.o: test_stub.c $(BPFOBJ)
 	$(call msg,CC,,$@)
 	$(CC) -c $(CFLAGS) -o $@ $<
 
-VMLINUX_BTF_PATHS := $(if $(O),$(O)/vmlinux)				\
+VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux)				\
 		     $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux)	\
 		     ../../../../vmlinux				\
 		     /sys/kernel/btf/vmlinux				\
 		     /boot/vmlinux-$(shell uname -r)
-VMLINUX_BTF := $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS))))
+VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS))))
 
 $(OUTPUT)/runqslower: $(BPFOBJ)
 	$(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/runqslower	\
@@ -182,8 +182,13 @@ $(BUILD_DIR)/libbpf $(BUILD_DIR)/bpftool $(INCLUDE_DIR):
 	mkdir -p $@
 
 $(INCLUDE_DIR)/vmlinux.h: $(VMLINUX_BTF) | $(BPFTOOL) $(INCLUDE_DIR)
+ifeq ($(VMLINUX_H),)
 	$(call msg,GEN,,$@)
 	$(BPFTOOL) btf dump file $(VMLINUX_BTF) format c > $@
+else
+	$(call msg,CP,,$@)
+	cp "$(VMLINUX_H)" $@
+endif
 
 # Get Clang's default includes on this system, as opposed to those seen by
 # '-target bpf'. This fixes "missing" files on some architectures/distros,
-- 
2.24.1


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH bpf-next 1/2] tools/bpftool: allow substituting custom vmlinux.h for the build
  2020-06-30  0:47 [PATCH bpf-next 1/2] tools/bpftool: allow substituting custom vmlinux.h for the build Andrii Nakryiko
  2020-06-30  0:47 ` [PATCH bpf-next 2/2] selftests/bpf: allow substituting custom vmlinux.h for selftests build Andrii Nakryiko
@ 2020-06-30  3:12 ` Yonghong Song
  2020-06-30 14:12 ` Daniel Borkmann
  2 siblings, 0 replies; 5+ messages in thread
From: Yonghong Song @ 2020-06-30  3:12 UTC (permalink / raw)
  To: Andrii Nakryiko, bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team



On 6/29/20 5:47 PM, Andrii Nakryiko wrote:
> In some build contexts (e.g., Travis CI build for outdated kernel), vmlinux.h,
> generated from available kernel, doesn't contain all the types necessary for
> BPF program compilation. For such set up, the most maintainable way to deal
> with this problem is to keep pre-generated (almost up-to-date) vmlinux.h
> checked in and use it for compilation purposes. bpftool after that can deal
> with kernel missing some of the features in runtime with no problems.
> 
> To that effect, allow to specify path to custom vmlinux.h to bpftool's
> Makefile with VMLINUX_H variable.
> 
> Signed-off-by: Andrii Nakryiko <andriin@fb.com>

Acked-by: Yonghong Song <yhs@fb.com>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH bpf-next 2/2] selftests/bpf: allow substituting custom vmlinux.h for selftests build
  2020-06-30  0:47 ` [PATCH bpf-next 2/2] selftests/bpf: allow substituting custom vmlinux.h for selftests build Andrii Nakryiko
@ 2020-06-30  4:51   ` Yonghong Song
  0 siblings, 0 replies; 5+ messages in thread
From: Yonghong Song @ 2020-06-30  4:51 UTC (permalink / raw)
  To: Andrii Nakryiko, bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team



On 6/29/20 5:47 PM, Andrii Nakryiko wrote:
> Similarly to bpftool Makefile, allow to specify custom location of vmlinux.h
> to be used during the build. This allows simpler testing setups with
> checked-in pre-generated vmlinux.h.
> 
> Signed-off-by: Andrii Nakryiko <andriin@fb.com>

Acked-by: Yonghong Song <yhs@fb.com>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH bpf-next 1/2] tools/bpftool: allow substituting custom vmlinux.h for the build
  2020-06-30  0:47 [PATCH bpf-next 1/2] tools/bpftool: allow substituting custom vmlinux.h for the build Andrii Nakryiko
  2020-06-30  0:47 ` [PATCH bpf-next 2/2] selftests/bpf: allow substituting custom vmlinux.h for selftests build Andrii Nakryiko
  2020-06-30  3:12 ` [PATCH bpf-next 1/2] tools/bpftool: allow substituting custom vmlinux.h for the build Yonghong Song
@ 2020-06-30 14:12 ` Daniel Borkmann
  2 siblings, 0 replies; 5+ messages in thread
From: Daniel Borkmann @ 2020-06-30 14:12 UTC (permalink / raw)
  To: Andrii Nakryiko, bpf, netdev, ast; +Cc: andrii.nakryiko, kernel-team

On 6/30/20 2:47 AM, Andrii Nakryiko wrote:
> In some build contexts (e.g., Travis CI build for outdated kernel), vmlinux.h,
> generated from available kernel, doesn't contain all the types necessary for
> BPF program compilation. For such set up, the most maintainable way to deal
> with this problem is to keep pre-generated (almost up-to-date) vmlinux.h
> checked in and use it for compilation purposes. bpftool after that can deal
> with kernel missing some of the features in runtime with no problems.
> 
> To that effect, allow to specify path to custom vmlinux.h to bpftool's
> Makefile with VMLINUX_H variable.
> 
> Signed-off-by: Andrii Nakryiko <andriin@fb.com>

Both applied, thanks!

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-30  0:47 [PATCH bpf-next 1/2] tools/bpftool: allow substituting custom vmlinux.h for the build Andrii Nakryiko
2020-06-30  0:47 ` [PATCH bpf-next 2/2] selftests/bpf: allow substituting custom vmlinux.h for selftests build Andrii Nakryiko
2020-06-30  4:51   ` Yonghong Song
2020-06-30  3:12 ` [PATCH bpf-next 1/2] tools/bpftool: allow substituting custom vmlinux.h for the build Yonghong Song
2020-06-30 14:12 ` Daniel Borkmann

BPF Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/bpf/0 bpf/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 bpf bpf/ https://lore.kernel.org/bpf \
		bpf@vger.kernel.org
	public-inbox-index bpf

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.bpf


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git