* [PATCH bpf-next 0/3] Use lightweigt version of bpftool
@ 2022-07-12 3:08 Pu Lehui
2022-07-12 3:08 ` [PATCH bpf-next 1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool Pu Lehui
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Pu Lehui @ 2022-07-12 3:08 UTC (permalink / raw)
To: bpf, netdev, linux-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Quentin Monnet, Martin KaFai Lau, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Jean-Philippe Brucker, Pu Lehui
Currently, samples/bpf, tools/runqslower and bpf/iterators use bpftool
for vmlinux.h, skeleton, and static linking only. We can use lightweight
bootstrap version of bpftool to handle these, and it will be faster.
Pu Lehui (3):
samples: bpf: Fix cross-compiling error by using bootstrap bpftool
tools: runqslower: build and use lightweight bootstrap version of
bpftool
bpf: iterators: build and use lightweight bootstrap version of bpftool
kernel/bpf/preload/iterators/Makefile | 13 +++++++++----
samples/bpf/Makefile | 16 +++++++++++-----
tools/bpf/runqslower/Makefile | 10 ++++++++--
3 files changed, 28 insertions(+), 11 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH bpf-next 1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool
2022-07-12 3:08 [PATCH bpf-next 0/3] Use lightweigt version of bpftool Pu Lehui
@ 2022-07-12 3:08 ` Pu Lehui
2022-07-12 10:11 ` Quentin Monnet
2022-07-12 3:08 ` [PATCH bpf-next 2/3] tools: runqslower: build and use lightweight bootstrap version of bpftool Pu Lehui
2022-07-12 3:08 ` [PATCH bpf-next 3/3] bpf: iterators: " Pu Lehui
2 siblings, 1 reply; 11+ messages in thread
From: Pu Lehui @ 2022-07-12 3:08 UTC (permalink / raw)
To: bpf, netdev, linux-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Quentin Monnet, Martin KaFai Lau, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Jean-Philippe Brucker, Pu Lehui
Currently, when cross compiling bpf samples, the host side cannot
use arch-specific bpftool to generate vmlinux.h or skeleton. Since
samples/bpf use bpftool for vmlinux.h, skeleton, and static linking
only, we can use lightweight bootstrap version of bpftool to handle
these, and it's always host-native.
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
---
samples/bpf/Makefile | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 5002a5b9a7da..57012b8259d2 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -282,12 +282,18 @@ $(LIBBPF): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
BPFTOOLDIR := $(TOOLS_PATH)/bpf/bpftool
BPFTOOL_OUTPUT := $(abspath $(BPF_SAMPLES_PATH))/bpftool
-BPFTOOL := $(BPFTOOL_OUTPUT)/bpftool
+BPFTOOL := $(BPFTOOL_OUTPUT)/bootstrap/bpftool
+ifeq ($(CROSS_COMPILE),)
$(BPFTOOL): $(LIBBPF) $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) | $(BPFTOOL_OUTPUT)
- $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
- OUTPUT=$(BPFTOOL_OUTPUT)/ \
- LIBBPF_OUTPUT=$(LIBBPF_OUTPUT)/ \
- LIBBPF_DESTDIR=$(LIBBPF_DESTDIR)/
+ $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
+ OUTPUT=$(BPFTOOL_OUTPUT)/ \
+ LIBBPF_BOOTSTRAP_OUTPUT=$(LIBBPF_OUTPUT)/ \
+ LIBBPF_BOOTSTRAP_DESTDIR=$(LIBBPF_DESTDIR)/ bootstrap
+else
+$(BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) | $(BPFTOOL_OUTPUT)
+ $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
+ OUTPUT=$(BPFTOOL_OUTPUT)/ bootstrap
+endif
$(LIBBPF_OUTPUT) $(BPFTOOL_OUTPUT):
$(call msg,MKDIR,$@)
--
2.25.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH bpf-next 2/3] tools: runqslower: build and use lightweight bootstrap version of bpftool
2022-07-12 3:08 [PATCH bpf-next 0/3] Use lightweigt version of bpftool Pu Lehui
2022-07-12 3:08 ` [PATCH bpf-next 1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool Pu Lehui
@ 2022-07-12 3:08 ` Pu Lehui
2022-07-13 18:52 ` Andrii Nakryiko
2022-07-12 3:08 ` [PATCH bpf-next 3/3] bpf: iterators: " Pu Lehui
2 siblings, 1 reply; 11+ messages in thread
From: Pu Lehui @ 2022-07-12 3:08 UTC (permalink / raw)
To: bpf, netdev, linux-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Quentin Monnet, Martin KaFai Lau, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Jean-Philippe Brucker, Pu Lehui
tools/runqslower use bpftool for vmlinux.h, skeleton, and static linking
only. So we can use lightweight bootstrap version of bpftool to handle
these, and it will be faster.
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
---
tools/bpf/runqslower/Makefile | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index da6de16a3dfb..8900c74f29e2 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -4,7 +4,7 @@ include ../../scripts/Makefile.include
OUTPUT ?= $(abspath .output)/
BPFTOOL_OUTPUT := $(OUTPUT)bpftool/
-DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)bpftool
+DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)bootstrap/bpftool
BPFTOOL ?= $(DEFAULT_BPFTOOL)
LIBBPF_SRC := $(abspath ../../lib/bpf)
BPFOBJ_OUTPUT := $(OUTPUT)libbpf/
@@ -86,6 +86,12 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OU
$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(BPFOBJ_OUTPUT) \
DESTDIR=$(BPFOBJ_OUTPUT) prefix= $(abspath $@) install_headers
+ifeq ($(CROSS_COMPILE),)
$(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BPFTOOL_OUTPUT)
$(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT) \
- ARCH= CROSS_COMPILE= CC=$(HOSTCC) LD=$(HOSTLD)
+ LIBBPF_BOOTSTRAP_OUTPUT=$(BPFOBJ_OUTPUT) \
+ LIBBPF_BOOTSTRAP_DESTDIR=$(BPF_DESTDIR) bootstrap
+else
+$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
+ $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT) bootstrap
+endif
--
2.25.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH bpf-next 3/3] bpf: iterators: build and use lightweight bootstrap version of bpftool
2022-07-12 3:08 [PATCH bpf-next 0/3] Use lightweigt version of bpftool Pu Lehui
2022-07-12 3:08 ` [PATCH bpf-next 1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool Pu Lehui
2022-07-12 3:08 ` [PATCH bpf-next 2/3] tools: runqslower: build and use lightweight bootstrap version of bpftool Pu Lehui
@ 2022-07-12 3:08 ` Pu Lehui
2022-07-13 18:55 ` Andrii Nakryiko
2 siblings, 1 reply; 11+ messages in thread
From: Pu Lehui @ 2022-07-12 3:08 UTC (permalink / raw)
To: bpf, netdev, linux-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Quentin Monnet, Martin KaFai Lau, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Jean-Philippe Brucker, Pu Lehui
kernel/bpf/preload/iterators use bpftool for vmlinux.h, skeleton, and
static linking only. So we can use lightweight bootstrap version of
bpftool to handle these, and it will be faster.
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
---
kernel/bpf/preload/iterators/Makefile | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/kernel/bpf/preload/iterators/Makefile b/kernel/bpf/preload/iterators/Makefile
index bfe24f8c5a20..cf5f39f95fed 100644
--- a/kernel/bpf/preload/iterators/Makefile
+++ b/kernel/bpf/preload/iterators/Makefile
@@ -9,7 +9,7 @@ LLVM_STRIP ?= llvm-strip
TOOLS_PATH := $(abspath ../../../../tools)
BPFTOOL_SRC := $(TOOLS_PATH)/bpf/bpftool
BPFTOOL_OUTPUT := $(abs_out)/bpftool
-DEFAULT_BPFTOOL := $(OUTPUT)/sbin/bpftool
+DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)/bootstrap/bpftool
BPFTOOL ?= $(DEFAULT_BPFTOOL)
LIBBPF_SRC := $(TOOLS_PATH)/lib/bpf
@@ -61,9 +61,14 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
OUTPUT=$(abspath $(dir $@))/ prefix= \
DESTDIR=$(LIBBPF_DESTDIR) $(abspath $@) install_headers
+ifeq ($(CROSS_COMPILE),)
$(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BPFTOOL_OUTPUT)
$(Q)$(MAKE) $(submake_extras) -C $(BPFTOOL_SRC) \
OUTPUT=$(BPFTOOL_OUTPUT)/ \
- LIBBPF_OUTPUT=$(LIBBPF_OUTPUT)/ \
- LIBBPF_DESTDIR=$(LIBBPF_DESTDIR)/ \
- prefix= DESTDIR=$(abs_out)/ install-bin
+ LIBBPF_BOOTSTRAP_OUTPUT=$(LIBBPF_OUTPUT)/ \
+ LIBBPF_BOOTSTRAP_DESTDIR=$(LIBBPF_DESTDIR)/ bootstrap
+else
+$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
+ $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOL_SRC) \
+ OUTPUT=$(BPFTOOL_OUTPUT)/ bootstrap
+endif
--
2.25.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH bpf-next 1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool
2022-07-12 3:08 ` [PATCH bpf-next 1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool Pu Lehui
@ 2022-07-12 10:11 ` Quentin Monnet
2022-07-12 11:32 ` Pu Lehui
0 siblings, 1 reply; 11+ messages in thread
From: Quentin Monnet @ 2022-07-12 10:11 UTC (permalink / raw)
To: Pu Lehui, bpf, netdev, linux-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Jean-Philippe Brucker
On 12/07/2022 04:08, Pu Lehui wrote:
> Currently, when cross compiling bpf samples, the host side cannot
> use arch-specific bpftool to generate vmlinux.h or skeleton. Since
> samples/bpf use bpftool for vmlinux.h, skeleton, and static linking
> only, we can use lightweight bootstrap version of bpftool to handle
> these, and it's always host-native.
>
> Signed-off-by: Pu Lehui <pulehui@huawei.com>
> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> ---
> samples/bpf/Makefile | 16 +++++++++++-----
> 1 file changed, 11 insertions(+), 5 deletions(-)
>
> diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
> index 5002a5b9a7da..57012b8259d2 100644
> --- a/samples/bpf/Makefile
> +++ b/samples/bpf/Makefile
> @@ -282,12 +282,18 @@ $(LIBBPF): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
>
> BPFTOOLDIR := $(TOOLS_PATH)/bpf/bpftool
> BPFTOOL_OUTPUT := $(abspath $(BPF_SAMPLES_PATH))/bpftool
> -BPFTOOL := $(BPFTOOL_OUTPUT)/bpftool
> +BPFTOOL := $(BPFTOOL_OUTPUT)/bootstrap/bpftool
> +ifeq ($(CROSS_COMPILE),)
> $(BPFTOOL): $(LIBBPF) $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) | $(BPFTOOL_OUTPUT)
> - $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
> - OUTPUT=$(BPFTOOL_OUTPUT)/ \
> - LIBBPF_OUTPUT=$(LIBBPF_OUTPUT)/ \
> - LIBBPF_DESTDIR=$(LIBBPF_DESTDIR)/
> + $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
> + OUTPUT=$(BPFTOOL_OUTPUT)/ \
> + LIBBPF_BOOTSTRAP_OUTPUT=$(LIBBPF_OUTPUT)/ \
> + LIBBPF_BOOTSTRAP_DESTDIR=$(LIBBPF_DESTDIR)/ bootstrap
> +else
> +$(BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) | $(BPFTOOL_OUTPUT)
Thanks for this! Just trying to fully understand the details here. When
cross-compiling, you leave aside the dependency on target-arch-libbpf,
so that "make -C <bpftool-dir> bootstrap" rebuilds its own host-arch
libbpf, is this correct?
> + $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
> + OUTPUT=$(BPFTOOL_OUTPUT)/ bootstrap
> +endif
>
> $(LIBBPF_OUTPUT) $(BPFTOOL_OUTPUT):
> $(call msg,MKDIR,$@)
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH bpf-next 1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool
2022-07-12 10:11 ` Quentin Monnet
@ 2022-07-12 11:32 ` Pu Lehui
2022-07-13 18:50 ` Andrii Nakryiko
0 siblings, 1 reply; 11+ messages in thread
From: Pu Lehui @ 2022-07-12 11:32 UTC (permalink / raw)
To: Quentin Monnet, bpf, netdev, linux-kernel
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Jean-Philippe Brucker
On 2022/7/12 18:11, Quentin Monnet wrote:
> On 12/07/2022 04:08, Pu Lehui wrote:
>> Currently, when cross compiling bpf samples, the host side cannot
>> use arch-specific bpftool to generate vmlinux.h or skeleton. Since
>> samples/bpf use bpftool for vmlinux.h, skeleton, and static linking
>> only, we can use lightweight bootstrap version of bpftool to handle
>> these, and it's always host-native.
>>
>> Signed-off-by: Pu Lehui <pulehui@huawei.com>
>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
>> ---
>> samples/bpf/Makefile | 16 +++++++++++-----
>> 1 file changed, 11 insertions(+), 5 deletions(-)
>>
>> diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
>> index 5002a5b9a7da..57012b8259d2 100644
>> --- a/samples/bpf/Makefile
>> +++ b/samples/bpf/Makefile
>> @@ -282,12 +282,18 @@ $(LIBBPF): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
>>
>> BPFTOOLDIR := $(TOOLS_PATH)/bpf/bpftool
>> BPFTOOL_OUTPUT := $(abspath $(BPF_SAMPLES_PATH))/bpftool
>> -BPFTOOL := $(BPFTOOL_OUTPUT)/bpftool
>> +BPFTOOL := $(BPFTOOL_OUTPUT)/bootstrap/bpftool
>> +ifeq ($(CROSS_COMPILE),)
>> $(BPFTOOL): $(LIBBPF) $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) | $(BPFTOOL_OUTPUT)
>> - $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
>> - OUTPUT=$(BPFTOOL_OUTPUT)/ \
>> - LIBBPF_OUTPUT=$(LIBBPF_OUTPUT)/ \
>> - LIBBPF_DESTDIR=$(LIBBPF_DESTDIR)/
>> + $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
>> + OUTPUT=$(BPFTOOL_OUTPUT)/ \
>> + LIBBPF_BOOTSTRAP_OUTPUT=$(LIBBPF_OUTPUT)/ \
>> + LIBBPF_BOOTSTRAP_DESTDIR=$(LIBBPF_DESTDIR)/ bootstrap
>> +else
>> +$(BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) | $(BPFTOOL_OUTPUT)
>
> Thanks for this! Just trying to fully understand the details here. When
> cross-compiling, you leave aside the dependency on target-arch-libbpf,
> so that "make -C <bpftool-dir> bootstrap" rebuilds its own host-arch
> libbpf, is this correct?
>
You're right. libbpf may does get out-of-sync. So the best way is to
compile both arch-specific libbpf simultaneously, and then attach to
bpftool. But it will make this job more complicated. Could we just add
back $(LIBBPF) to handle this?
>> + $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
>> + OUTPUT=$(BPFTOOL_OUTPUT)/ bootstrap
>> +endif
>>
>> $(LIBBPF_OUTPUT) $(BPFTOOL_OUTPUT):
>> $(call msg,MKDIR,$@)
>
> .
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH bpf-next 1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool
2022-07-12 11:32 ` Pu Lehui
@ 2022-07-13 18:50 ` Andrii Nakryiko
0 siblings, 0 replies; 11+ messages in thread
From: Andrii Nakryiko @ 2022-07-13 18:50 UTC (permalink / raw)
To: Pu Lehui
Cc: Quentin Monnet, bpf, Networking, open list, Alexei Starovoitov,
Daniel Borkmann, Andrii Nakryiko, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Jean-Philippe Brucker
On Tue, Jul 12, 2022 at 4:32 AM Pu Lehui <pulehui@huawei.com> wrote:
>
>
>
> On 2022/7/12 18:11, Quentin Monnet wrote:
> > On 12/07/2022 04:08, Pu Lehui wrote:
> >> Currently, when cross compiling bpf samples, the host side cannot
> >> use arch-specific bpftool to generate vmlinux.h or skeleton. Since
> >> samples/bpf use bpftool for vmlinux.h, skeleton, and static linking
> >> only, we can use lightweight bootstrap version of bpftool to handle
> >> these, and it's always host-native.
> >>
> >> Signed-off-by: Pu Lehui <pulehui@huawei.com>
> >> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> >> ---
> >> samples/bpf/Makefile | 16 +++++++++++-----
> >> 1 file changed, 11 insertions(+), 5 deletions(-)
> >>
> >> diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
> >> index 5002a5b9a7da..57012b8259d2 100644
> >> --- a/samples/bpf/Makefile
> >> +++ b/samples/bpf/Makefile
> >> @@ -282,12 +282,18 @@ $(LIBBPF): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
> >>
> >> BPFTOOLDIR := $(TOOLS_PATH)/bpf/bpftool
> >> BPFTOOL_OUTPUT := $(abspath $(BPF_SAMPLES_PATH))/bpftool
> >> -BPFTOOL := $(BPFTOOL_OUTPUT)/bpftool
> >> +BPFTOOL := $(BPFTOOL_OUTPUT)/bootstrap/bpftool
> >> +ifeq ($(CROSS_COMPILE),)
> >> $(BPFTOOL): $(LIBBPF) $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) | $(BPFTOOL_OUTPUT)
> >> - $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
> >> - OUTPUT=$(BPFTOOL_OUTPUT)/ \
> >> - LIBBPF_OUTPUT=$(LIBBPF_OUTPUT)/ \
> >> - LIBBPF_DESTDIR=$(LIBBPF_DESTDIR)/
> >> + $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
> >> + OUTPUT=$(BPFTOOL_OUTPUT)/ \
> >> + LIBBPF_BOOTSTRAP_OUTPUT=$(LIBBPF_OUTPUT)/ \
> >> + LIBBPF_BOOTSTRAP_DESTDIR=$(LIBBPF_DESTDIR)/ bootstrap
> >> +else
> >> +$(BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) | $(BPFTOOL_OUTPUT)
> >
> > Thanks for this! Just trying to fully understand the details here. When
> > cross-compiling, you leave aside the dependency on target-arch-libbpf,
> > so that "make -C <bpftool-dir> bootstrap" rebuilds its own host-arch
> > libbpf, is this correct?
> >
>
> You're right. libbpf may does get out-of-sync. So the best way is to
> compile both arch-specific libbpf simultaneously, and then attach to
> bpftool. But it will make this job more complicated. Could we just add
> back $(LIBBPF) to handle this?
>
Maybe let's keep it simple and let bpftool's Makefile deal with
cross-compile issue and building its own libbpf? So just request
bootstrap, but not try to share libbpf between samples/bpf and
bpftool? Especially that is this "samples", such complexity in
Makefile seems like a micro-optimization.
> >> + $(MAKE) -C $(BPFTOOLDIR) srctree=$(BPF_SAMPLES_PATH)/../../ \
> >> + OUTPUT=$(BPFTOOL_OUTPUT)/ bootstrap
> >> +endif
> >>
> >> $(LIBBPF_OUTPUT) $(BPFTOOL_OUTPUT):
> >> $(call msg,MKDIR,$@)
> >
> > .
> >
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH bpf-next 2/3] tools: runqslower: build and use lightweight bootstrap version of bpftool
2022-07-12 3:08 ` [PATCH bpf-next 2/3] tools: runqslower: build and use lightweight bootstrap version of bpftool Pu Lehui
@ 2022-07-13 18:52 ` Andrii Nakryiko
2022-07-14 2:02 ` Pu Lehui
0 siblings, 1 reply; 11+ messages in thread
From: Andrii Nakryiko @ 2022-07-13 18:52 UTC (permalink / raw)
To: Pu Lehui
Cc: bpf, Networking, open list, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Quentin Monnet, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Jean-Philippe Brucker
On Mon, Jul 11, 2022 at 7:37 PM Pu Lehui <pulehui@huawei.com> wrote:
>
> tools/runqslower use bpftool for vmlinux.h, skeleton, and static linking
> only. So we can use lightweight bootstrap version of bpftool to handle
> these, and it will be faster.
>
> Signed-off-by: Pu Lehui <pulehui@huawei.com>
> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> ---
> tools/bpf/runqslower/Makefile | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
> index da6de16a3dfb..8900c74f29e2 100644
> --- a/tools/bpf/runqslower/Makefile
> +++ b/tools/bpf/runqslower/Makefile
> @@ -4,7 +4,7 @@ include ../../scripts/Makefile.include
> OUTPUT ?= $(abspath .output)/
>
> BPFTOOL_OUTPUT := $(OUTPUT)bpftool/
> -DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)bpftool
> +DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)bootstrap/bpftool
> BPFTOOL ?= $(DEFAULT_BPFTOOL)
> LIBBPF_SRC := $(abspath ../../lib/bpf)
> BPFOBJ_OUTPUT := $(OUTPUT)libbpf/
> @@ -86,6 +86,12 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OU
> $(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(BPFOBJ_OUTPUT) \
> DESTDIR=$(BPFOBJ_OUTPUT) prefix= $(abspath $@) install_headers
>
> +ifeq ($(CROSS_COMPILE),)
> $(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BPFTOOL_OUTPUT)
> $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT) \
> - ARCH= CROSS_COMPILE= CC=$(HOSTCC) LD=$(HOSTLD)
> + LIBBPF_BOOTSTRAP_OUTPUT=$(BPFOBJ_OUTPUT) \
> + LIBBPF_BOOTSTRAP_DESTDIR=$(BPF_DESTDIR) bootstrap
> +else
> +$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
> + $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT) bootstrap
> +endif
Same comment as on the other patch, this CROSS_COMPILE if/else seems a
bit fragile, let's keep only the second cleaner part, maybe?
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH bpf-next 3/3] bpf: iterators: build and use lightweight bootstrap version of bpftool
2022-07-12 3:08 ` [PATCH bpf-next 3/3] bpf: iterators: " Pu Lehui
@ 2022-07-13 18:55 ` Andrii Nakryiko
2022-07-13 19:03 ` Quentin Monnet
0 siblings, 1 reply; 11+ messages in thread
From: Andrii Nakryiko @ 2022-07-13 18:55 UTC (permalink / raw)
To: Pu Lehui
Cc: bpf, Networking, open list, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Quentin Monnet, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Jean-Philippe Brucker
On Mon, Jul 11, 2022 at 7:37 PM Pu Lehui <pulehui@huawei.com> wrote:
>
> kernel/bpf/preload/iterators use bpftool for vmlinux.h, skeleton, and
> static linking only. So we can use lightweight bootstrap version of
> bpftool to handle these, and it will be faster.
>
> Signed-off-by: Pu Lehui <pulehui@huawei.com>
> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> ---
> kernel/bpf/preload/iterators/Makefile | 13 +++++++++----
> 1 file changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/kernel/bpf/preload/iterators/Makefile b/kernel/bpf/preload/iterators/Makefile
> index bfe24f8c5a20..cf5f39f95fed 100644
> --- a/kernel/bpf/preload/iterators/Makefile
> +++ b/kernel/bpf/preload/iterators/Makefile
> @@ -9,7 +9,7 @@ LLVM_STRIP ?= llvm-strip
> TOOLS_PATH := $(abspath ../../../../tools)
> BPFTOOL_SRC := $(TOOLS_PATH)/bpf/bpftool
> BPFTOOL_OUTPUT := $(abs_out)/bpftool
> -DEFAULT_BPFTOOL := $(OUTPUT)/sbin/bpftool
> +DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)/bootstrap/bpftool
> BPFTOOL ?= $(DEFAULT_BPFTOOL)
>
> LIBBPF_SRC := $(TOOLS_PATH)/lib/bpf
> @@ -61,9 +61,14 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
> OUTPUT=$(abspath $(dir $@))/ prefix= \
> DESTDIR=$(LIBBPF_DESTDIR) $(abspath $@) install_headers
>
> +ifeq ($(CROSS_COMPILE),)
> $(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BPFTOOL_OUTPUT)
> $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOL_SRC) \
> OUTPUT=$(BPFTOOL_OUTPUT)/ \
> - LIBBPF_OUTPUT=$(LIBBPF_OUTPUT)/ \
> - LIBBPF_DESTDIR=$(LIBBPF_DESTDIR)/ \
> - prefix= DESTDIR=$(abs_out)/ install-bin
> + LIBBPF_BOOTSTRAP_OUTPUT=$(LIBBPF_OUTPUT)/ \
> + LIBBPF_BOOTSTRAP_DESTDIR=$(LIBBPF_DESTDIR)/ bootstrap
> +else
> +$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
> + $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOL_SRC) \
> + OUTPUT=$(BPFTOOL_OUTPUT)/ bootstrap
> +endif
another idea (related to my two previous comments for this patch set),
maybe we can teach bpftool's Makefile to reuse LIBBPF_OUTPUT as
LIBBPF_BOOTSTRAP_OUTPUT, if there is no CROSS_COMPILE? Then we can
keep iterators/Makefile, samples/bpf/Makefile and runqslower/Makefile
simpler and ignorant of CROSS_COMPILE, but still get the benefit of
not rebuilding libbpf unnecessarily in non-cross-compile mode?
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH bpf-next 3/3] bpf: iterators: build and use lightweight bootstrap version of bpftool
2022-07-13 18:55 ` Andrii Nakryiko
@ 2022-07-13 19:03 ` Quentin Monnet
0 siblings, 0 replies; 11+ messages in thread
From: Quentin Monnet @ 2022-07-13 19:03 UTC (permalink / raw)
To: Andrii Nakryiko, Pu Lehui
Cc: bpf, Networking, open list, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Jean-Philippe Brucker
On 13/07/2022 19:55, Andrii Nakryiko wrote:
> On Mon, Jul 11, 2022 at 7:37 PM Pu Lehui <pulehui@huawei.com> wrote:
>>
>> kernel/bpf/preload/iterators use bpftool for vmlinux.h, skeleton, and
>> static linking only. So we can use lightweight bootstrap version of
>> bpftool to handle these, and it will be faster.
>>
>> Signed-off-by: Pu Lehui <pulehui@huawei.com>
>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
>> ---
>> kernel/bpf/preload/iterators/Makefile | 13 +++++++++----
>> 1 file changed, 9 insertions(+), 4 deletions(-)
>>
>> diff --git a/kernel/bpf/preload/iterators/Makefile b/kernel/bpf/preload/iterators/Makefile
>> index bfe24f8c5a20..cf5f39f95fed 100644
>> --- a/kernel/bpf/preload/iterators/Makefile
>> +++ b/kernel/bpf/preload/iterators/Makefile
>> @@ -9,7 +9,7 @@ LLVM_STRIP ?= llvm-strip
>> TOOLS_PATH := $(abspath ../../../../tools)
>> BPFTOOL_SRC := $(TOOLS_PATH)/bpf/bpftool
>> BPFTOOL_OUTPUT := $(abs_out)/bpftool
>> -DEFAULT_BPFTOOL := $(OUTPUT)/sbin/bpftool
>> +DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)/bootstrap/bpftool
>> BPFTOOL ?= $(DEFAULT_BPFTOOL)
>>
>> LIBBPF_SRC := $(TOOLS_PATH)/lib/bpf
>> @@ -61,9 +61,14 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
>> OUTPUT=$(abspath $(dir $@))/ prefix= \
>> DESTDIR=$(LIBBPF_DESTDIR) $(abspath $@) install_headers
>>
>> +ifeq ($(CROSS_COMPILE),)
>> $(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BPFTOOL_OUTPUT)
>> $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOL_SRC) \
>> OUTPUT=$(BPFTOOL_OUTPUT)/ \
>> - LIBBPF_OUTPUT=$(LIBBPF_OUTPUT)/ \
>> - LIBBPF_DESTDIR=$(LIBBPF_DESTDIR)/ \
>> - prefix= DESTDIR=$(abs_out)/ install-bin
>> + LIBBPF_BOOTSTRAP_OUTPUT=$(LIBBPF_OUTPUT)/ \
>> + LIBBPF_BOOTSTRAP_DESTDIR=$(LIBBPF_DESTDIR)/ bootstrap
>> +else
>> +$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
>> + $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOL_SRC) \
>> + OUTPUT=$(BPFTOOL_OUTPUT)/ bootstrap
>> +endif
>
> another idea (related to my two previous comments for this patch set),
> maybe we can teach bpftool's Makefile to reuse LIBBPF_OUTPUT as
> LIBBPF_BOOTSTRAP_OUTPUT, if there is no CROSS_COMPILE? Then we can
> keep iterators/Makefile, samples/bpf/Makefile and runqslower/Makefile
> simpler and ignorant of CROSS_COMPILE, but still get the benefit of
> not rebuilding libbpf unnecessarily in non-cross-compile mode?
Could be a good idea. Seeing how the HID BPF patches add BTF/skeletons
generation at new locations, I'm also starting to wonder if it would be
worth having a Makefile.bpftool.include of some sort to harmonise the
way we compile the bootstrap bpftool as a dependency, and make it easier
to maintain. I haven't looked at how feasible that would be, yet.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH bpf-next 2/3] tools: runqslower: build and use lightweight bootstrap version of bpftool
2022-07-13 18:52 ` Andrii Nakryiko
@ 2022-07-14 2:02 ` Pu Lehui
0 siblings, 0 replies; 11+ messages in thread
From: Pu Lehui @ 2022-07-14 2:02 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: bpf, Networking, open list, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Quentin Monnet, Martin KaFai Lau, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Jean-Philippe Brucker
On 2022/7/14 2:52, Andrii Nakryiko wrote:
> On Mon, Jul 11, 2022 at 7:37 PM Pu Lehui <pulehui@huawei.com> wrote:
>>
>> tools/runqslower use bpftool for vmlinux.h, skeleton, and static linking
>> only. So we can use lightweight bootstrap version of bpftool to handle
>> these, and it will be faster.
>>
>> Signed-off-by: Pu Lehui <pulehui@huawei.com>
>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
>> ---
>> tools/bpf/runqslower/Makefile | 10 ++++++++--
>> 1 file changed, 8 insertions(+), 2 deletions(-)
>>
>> diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
>> index da6de16a3dfb..8900c74f29e2 100644
>> --- a/tools/bpf/runqslower/Makefile
>> +++ b/tools/bpf/runqslower/Makefile
>> @@ -4,7 +4,7 @@ include ../../scripts/Makefile.include
>> OUTPUT ?= $(abspath .output)/
>>
>> BPFTOOL_OUTPUT := $(OUTPUT)bpftool/
>> -DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)bpftool
>> +DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)bootstrap/bpftool
>> BPFTOOL ?= $(DEFAULT_BPFTOOL)
>> LIBBPF_SRC := $(abspath ../../lib/bpf)
>> BPFOBJ_OUTPUT := $(OUTPUT)libbpf/
>> @@ -86,6 +86,12 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OU
>> $(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(BPFOBJ_OUTPUT) \
>> DESTDIR=$(BPFOBJ_OUTPUT) prefix= $(abspath $@) install_headers
>>
>> +ifeq ($(CROSS_COMPILE),)
>> $(DEFAULT_BPFTOOL): $(BPFOBJ) | $(BPFTOOL_OUTPUT)
>> $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT) \
>> - ARCH= CROSS_COMPILE= CC=$(HOSTCC) LD=$(HOSTLD)
>> + LIBBPF_BOOTSTRAP_OUTPUT=$(BPFOBJ_OUTPUT) \
>> + LIBBPF_BOOTSTRAP_DESTDIR=$(BPF_DESTDIR) bootstrap
>> +else
>> +$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
>> + $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT) bootstrap
>> +endif
>
> Same comment as on the other patch, this CROSS_COMPILE if/else seems a
> bit fragile, let's keep only the second cleaner part, maybe?
>
I will make it simple.
>
>> --
>> 2.25.1
>>
> .
>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2022-07-14 2:02 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-12 3:08 [PATCH bpf-next 0/3] Use lightweigt version of bpftool Pu Lehui
2022-07-12 3:08 ` [PATCH bpf-next 1/3] samples: bpf: Fix cross-compiling error by using bootstrap bpftool Pu Lehui
2022-07-12 10:11 ` Quentin Monnet
2022-07-12 11:32 ` Pu Lehui
2022-07-13 18:50 ` Andrii Nakryiko
2022-07-12 3:08 ` [PATCH bpf-next 2/3] tools: runqslower: build and use lightweight bootstrap version of bpftool Pu Lehui
2022-07-13 18:52 ` Andrii Nakryiko
2022-07-14 2:02 ` Pu Lehui
2022-07-12 3:08 ` [PATCH bpf-next 3/3] bpf: iterators: " Pu Lehui
2022-07-13 18:55 ` Andrii Nakryiko
2022-07-13 19:03 ` Quentin Monnet
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.