linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing
@ 2023-02-01  1:50 Ian Rogers
  2023-02-01  9:59 ` Jiri Olsa
  0 siblings, 1 reply; 9+ messages in thread
From: Ian Rogers @ 2023-02-01  1:50 UTC (permalink / raw)
  To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
	KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, Connor OBrien,
	Nathan Chancellor, Ian Rogers, Kumar Kartikeya Dwivedi, bpf,
	linux-kernel

Avoid passing CROSS_COMPILE to submakes and ensure CFLAGS is forced to
HOSTCFLAGS for submake builds. This fixes problems with cross
compilation.

Tidy to not unnecessarily modify/export CFLAGS, make the override for
prepare and build clearer.

Fixes: 13e07691a16f ("tools/resolve_btfids: Alter how HOSTCC is forced")
Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/bpf/resolve_btfids/Makefile | 49 ++++++++++++++++---------------
 1 file changed, 26 insertions(+), 23 deletions(-)

diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
index daed388aa5d7..c9b6cf1fb844 100644
--- a/tools/bpf/resolve_btfids/Makefile
+++ b/tools/bpf/resolve_btfids/Makefile
@@ -17,12 +17,7 @@ else
   MAKEFLAGS=--no-print-directory
 endif
 
-# always use the host compiler
-HOST_OVERRIDES := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" ARCH="$(HOSTARCH)" \
-		  EXTRA_CFLAGS="$(HOSTCFLAGS) $(KBUILD_HOSTCFLAGS)"
-
 RM      ?= rm
-CROSS_COMPILE =
 
 OUTPUT ?= $(srctree)/tools/bpf/resolve_btfids/
 
@@ -43,6 +38,29 @@ SUBCMD_INCLUDE := $(SUBCMD_DESTDIR)include
 BINARY     := $(OUTPUT)/resolve_btfids
 BINARY_IN  := $(BINARY)-in.o
 
+LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
+LIBELF_LIBS  := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
+
+RESOLVE_BTFIDS_CFLAGS = -g \
+          -I$(srctree)/tools/include \
+          -I$(srctree)/tools/include/uapi \
+          -I$(LIBBPF_INCLUDE) \
+          -I$(SUBCMD_INCLUDE) \
+          $(LIBELF_FLAGS)
+
+# Overrides for the prepare step libraries.
+HOST_OVERRIDES_PREPARE := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" \
+	  ARCH="$(HOSTARCH)" CROSS_COMPILE=""
+
+# Overrides for Makefile.build C targets.
+HOST_OVERRIDES_BUILD := $(HOST_OVERRIDES_PREPARE) \
+	  CFLAGS="$(HOSTCFLAGS) $(KBUILD_HOSTCFLAGS) $(RESOLVE_BTFIDS_CFLAGS)" \
+
+LIBS = $(LIBELF_LIBS) -lz
+
+export srctree OUTPUT Q
+include $(srctree)/tools/build/Makefile.include
+
 all: $(BINARY)
 
 prepare: $(BPFOBJ) $(SUBCMDOBJ)
@@ -53,31 +71,16 @@ $(OUTPUT) $(OUTPUT)/libsubcmd $(LIBBPF_OUT):
 
 $(SUBCMDOBJ): fixdep FORCE | $(OUTPUT)/libsubcmd
 	$(Q)$(MAKE) -C $(SUBCMD_SRC) OUTPUT=$(SUBCMD_OUT) \
-		    DESTDIR=$(SUBCMD_DESTDIR) $(HOST_OVERRIDES) prefix= subdir= \
+		    DESTDIR=$(SUBCMD_DESTDIR) $(HOST_OVERRIDES_PREPARE) prefix= subdir= \
 		    $(abspath $@) install_headers
 
 $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OUT)
 	$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(LIBBPF_OUT)    \
-		    DESTDIR=$(LIBBPF_DESTDIR) $(HOST_OVERRIDES) prefix= subdir= \
+		    DESTDIR=$(LIBBPF_DESTDIR) $(HOST_OVERRIDES_PREPARE) prefix= subdir= \
 		    $(abspath $@) install_headers
 
-LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
-LIBELF_LIBS  := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
-
-CFLAGS += -g \
-          -I$(srctree)/tools/include \
-          -I$(srctree)/tools/include/uapi \
-          -I$(LIBBPF_INCLUDE) \
-          -I$(SUBCMD_INCLUDE) \
-          $(LIBELF_FLAGS)
-
-LIBS = $(LIBELF_LIBS) -lz
-
-export srctree OUTPUT CFLAGS Q
-include $(srctree)/tools/build/Makefile.include
-
 $(BINARY_IN): fixdep FORCE prepare | $(OUTPUT)
-	$(Q)$(MAKE) $(build)=resolve_btfids $(HOST_OVERRIDES)
+	$(Q)$(MAKE) $(build)=resolve_btfids $(HOST_OVERRIDES_BUILD)
 
 $(BINARY): $(BPFOBJ) $(SUBCMDOBJ) $(BINARY_IN)
 	$(call msg,LINK,$@)
-- 
2.39.1.456.gfc5497dd1b-goog


^ permalink raw reply related	[flat|nested] 9+ messages in thread
* Re: [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing
  2023-02-01  1:50 [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing Ian Rogers
@ 2023-02-01  9:59 ` Jiri Olsa
  2023-02-01 10:43   ` Jiri Olsa
  0 siblings, 1 reply; 9+ messages in thread
From: Jiri Olsa @ 2023-02-01  9:59 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
	KP Singh, Stanislav Fomichev, Hao Luo, Connor OBrien,
	Nathan Chancellor, Kumar Kartikeya Dwivedi, bpf, linux-kernel

On Tue, Jan 31, 2023 at 05:50:15PM -0800, Ian Rogers wrote:
> Avoid passing CROSS_COMPILE to submakes and ensure CFLAGS is forced to
> HOSTCFLAGS for submake builds. This fixes problems with cross
> compilation.
> 
> Tidy to not unnecessarily modify/export CFLAGS, make the override for
> prepare and build clearer.
> 
> Fixes: 13e07691a16f ("tools/resolve_btfids: Alter how HOSTCC is forced")
> Reported-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Ian Rogers <irogers@google.com>

hum, that seems to build just the fixdep and skip the resolve_btfids binary

make[1]: Entering directory '/home/jolsa/kernel/linux-qemu/build'
  GEN     Makefile
  CALL    ../scripts/checksyscalls.sh
  DESCEND bpf/resolve_btfids
  HOSTCC  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep.o
  HOSTLD  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep-in.o
  LINK    /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep
  UPD     include/generated/utsversion.h
  CC      init/version-timestamp.o
  LD      .tmp_vmlinux.btf
  BTF     .btf.vmlinux.bin.o
die__process_unit: DW_TAG_label (0xa) @ <0x4f0d4> not handled!

  ...

die__process_unit: DW_TAG_label (0xa) @ <0xaf91cc3> not handled!
die__process_unit: DW_TAG_label (0xa) @ <0xb032fa7> not handled!
  LD      .tmp_vmlinux.kallsyms1
  NM      .tmp_vmlinux.kallsyms1.syms
  KSYMS   .tmp_vmlinux.kallsyms1.S
  AS      .tmp_vmlinux.kallsyms1.S
  LD      .tmp_vmlinux.kallsyms2
  NM      .tmp_vmlinux.kallsyms2.syms
  KSYMS   .tmp_vmlinux.kallsyms2.S
  AS      .tmp_vmlinux.kallsyms2.S
  LD      .tmp_vmlinux.kallsyms3
  NM      .tmp_vmlinux.kallsyms3.syms
  KSYMS   .tmp_vmlinux.kallsyms3.S
  AS      .tmp_vmlinux.kallsyms3.S
  LD      vmlinux
  BTFIDS  vmlinux
../scripts/link-vmlinux.sh: line 277: ./tools/bpf/resolve_btfids/resolve_btfids: No such file or directory
make[2]: *** [../scripts/Makefile.vmlinux:35: vmlinux] Error 127
make[2]: *** Deleting file 'vmlinux'
make[1]: *** [/home/jolsa/kernel/linux-qemu/Makefile:1264: vmlinux] Error 2
make[1]: Leaving directory '/home/jolsa/kernel/linux-qemu/build'
make: *** [Makefile:242: __sub-make] Error 2

we actually have the hostprogs support in tools/build and we use it for
fixdep, I think we should use it also here, I'll check

jirka

> ---
>  tools/bpf/resolve_btfids/Makefile | 49 ++++++++++++++++---------------
>  1 file changed, 26 insertions(+), 23 deletions(-)
> 
> diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
> index daed388aa5d7..c9b6cf1fb844 100644
> --- a/tools/bpf/resolve_btfids/Makefile
> +++ b/tools/bpf/resolve_btfids/Makefile
> @@ -17,12 +17,7 @@ else
>    MAKEFLAGS=--no-print-directory
>  endif
>  
> -# always use the host compiler
> -HOST_OVERRIDES := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" ARCH="$(HOSTARCH)" \
> -		  EXTRA_CFLAGS="$(HOSTCFLAGS) $(KBUILD_HOSTCFLAGS)"
> -
>  RM      ?= rm
> -CROSS_COMPILE =
>  
>  OUTPUT ?= $(srctree)/tools/bpf/resolve_btfids/
>  
> @@ -43,6 +38,29 @@ SUBCMD_INCLUDE := $(SUBCMD_DESTDIR)include
>  BINARY     := $(OUTPUT)/resolve_btfids
>  BINARY_IN  := $(BINARY)-in.o
>  
> +LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
> +LIBELF_LIBS  := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
> +
> +RESOLVE_BTFIDS_CFLAGS = -g \
> +          -I$(srctree)/tools/include \
> +          -I$(srctree)/tools/include/uapi \
> +          -I$(LIBBPF_INCLUDE) \
> +          -I$(SUBCMD_INCLUDE) \
> +          $(LIBELF_FLAGS)
> +
> +# Overrides for the prepare step libraries.
> +HOST_OVERRIDES_PREPARE := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" \
> +	  ARCH="$(HOSTARCH)" CROSS_COMPILE=""
> +
> +# Overrides for Makefile.build C targets.
> +HOST_OVERRIDES_BUILD := $(HOST_OVERRIDES_PREPARE) \
> +	  CFLAGS="$(HOSTCFLAGS) $(KBUILD_HOSTCFLAGS) $(RESOLVE_BTFIDS_CFLAGS)" \
> +
> +LIBS = $(LIBELF_LIBS) -lz
> +
> +export srctree OUTPUT Q
> +include $(srctree)/tools/build/Makefile.include
> +
>  all: $(BINARY)
>  
>  prepare: $(BPFOBJ) $(SUBCMDOBJ)
> @@ -53,31 +71,16 @@ $(OUTPUT) $(OUTPUT)/libsubcmd $(LIBBPF_OUT):
>  
>  $(SUBCMDOBJ): fixdep FORCE | $(OUTPUT)/libsubcmd
>  	$(Q)$(MAKE) -C $(SUBCMD_SRC) OUTPUT=$(SUBCMD_OUT) \
> -		    DESTDIR=$(SUBCMD_DESTDIR) $(HOST_OVERRIDES) prefix= subdir= \
> +		    DESTDIR=$(SUBCMD_DESTDIR) $(HOST_OVERRIDES_PREPARE) prefix= subdir= \
>  		    $(abspath $@) install_headers
>  
>  $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OUT)
>  	$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(LIBBPF_OUT)    \
> -		    DESTDIR=$(LIBBPF_DESTDIR) $(HOST_OVERRIDES) prefix= subdir= \
> +		    DESTDIR=$(LIBBPF_DESTDIR) $(HOST_OVERRIDES_PREPARE) prefix= subdir= \
>  		    $(abspath $@) install_headers
>  
> -LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
> -LIBELF_LIBS  := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
> -
> -CFLAGS += -g \
> -          -I$(srctree)/tools/include \
> -          -I$(srctree)/tools/include/uapi \
> -          -I$(LIBBPF_INCLUDE) \
> -          -I$(SUBCMD_INCLUDE) \
> -          $(LIBELF_FLAGS)
> -
> -LIBS = $(LIBELF_LIBS) -lz
> -
> -export srctree OUTPUT CFLAGS Q
> -include $(srctree)/tools/build/Makefile.include
> -
>  $(BINARY_IN): fixdep FORCE prepare | $(OUTPUT)
> -	$(Q)$(MAKE) $(build)=resolve_btfids $(HOST_OVERRIDES)
> +	$(Q)$(MAKE) $(build)=resolve_btfids $(HOST_OVERRIDES_BUILD)
>  
>  $(BINARY): $(BPFOBJ) $(SUBCMDOBJ) $(BINARY_IN)
>  	$(call msg,LINK,$@)
> -- 
> 2.39.1.456.gfc5497dd1b-goog
> 

^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing
  2023-02-01  9:59 ` Jiri Olsa
@ 2023-02-01 10:43   ` Jiri Olsa
  2023-02-01 18:57     ` Ian Rogers
  0 siblings, 1 reply; 9+ messages in thread
From: Jiri Olsa @ 2023-02-01 10:43 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
	KP Singh, Stanislav Fomichev, Hao Luo, Connor OBrien,
	Nathan Chancellor, Kumar Kartikeya Dwivedi, bpf, linux-kernel

On Wed, Feb 01, 2023 at 11:00:02AM +0100, Jiri Olsa wrote:
> On Tue, Jan 31, 2023 at 05:50:15PM -0800, Ian Rogers wrote:
> > Avoid passing CROSS_COMPILE to submakes and ensure CFLAGS is forced to
> > HOSTCFLAGS for submake builds. This fixes problems with cross
> > compilation.
> > 
> > Tidy to not unnecessarily modify/export CFLAGS, make the override for
> > prepare and build clearer.
> > 
> > Fixes: 13e07691a16f ("tools/resolve_btfids: Alter how HOSTCC is forced")
> > Reported-by: Nathan Chancellor <nathan@kernel.org>
> > Signed-off-by: Ian Rogers <irogers@google.com>
> 
> hum, that seems to build just the fixdep and skip the resolve_btfids binary
> 
> make[1]: Entering directory '/home/jolsa/kernel/linux-qemu/build'
>   GEN     Makefile
>   CALL    ../scripts/checksyscalls.sh
>   DESCEND bpf/resolve_btfids
>   HOSTCC  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep.o
>   HOSTLD  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep-in.o
>   LINK    /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep
>   UPD     include/generated/utsversion.h
>   CC      init/version-timestamp.o
>   LD      .tmp_vmlinux.btf
>   BTF     .btf.vmlinux.bin.o
> die__process_unit: DW_TAG_label (0xa) @ <0x4f0d4> not handled!
> 
>   ...
> 
> die__process_unit: DW_TAG_label (0xa) @ <0xaf91cc3> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0xb032fa7> not handled!
>   LD      .tmp_vmlinux.kallsyms1
>   NM      .tmp_vmlinux.kallsyms1.syms
>   KSYMS   .tmp_vmlinux.kallsyms1.S
>   AS      .tmp_vmlinux.kallsyms1.S
>   LD      .tmp_vmlinux.kallsyms2
>   NM      .tmp_vmlinux.kallsyms2.syms
>   KSYMS   .tmp_vmlinux.kallsyms2.S
>   AS      .tmp_vmlinux.kallsyms2.S
>   LD      .tmp_vmlinux.kallsyms3
>   NM      .tmp_vmlinux.kallsyms3.syms
>   KSYMS   .tmp_vmlinux.kallsyms3.S
>   AS      .tmp_vmlinux.kallsyms3.S
>   LD      vmlinux
>   BTFIDS  vmlinux
> ../scripts/link-vmlinux.sh: line 277: ./tools/bpf/resolve_btfids/resolve_btfids: No such file or directory
> make[2]: *** [../scripts/Makefile.vmlinux:35: vmlinux] Error 127
> make[2]: *** Deleting file 'vmlinux'
> make[1]: *** [/home/jolsa/kernel/linux-qemu/Makefile:1264: vmlinux] Error 2
> make[1]: Leaving directory '/home/jolsa/kernel/linux-qemu/build'
> make: *** [Makefile:242: __sub-make] Error 2
> 
> we actually have the hostprogs support in tools/build and we use it for
> fixdep, I think we should use it also here, I'll check

it doesn't look that bad.. the change below fixes the build for me,
perhaps we should do that for all the host tools

jirka


---
diff --git a/tools/bpf/resolve_btfids/Build b/tools/bpf/resolve_btfids/Build
index ae82da03f9bf..077de3829c72 100644
--- a/tools/bpf/resolve_btfids/Build
+++ b/tools/bpf/resolve_btfids/Build
@@ -1,3 +1,5 @@
+hostprogs := resolve_btfids
+
 resolve_btfids-y += main.o
 resolve_btfids-y += rbtree.o
 resolve_btfids-y += zalloc.o
@@ -7,4 +9,4 @@ resolve_btfids-y += str_error_r.o
 
 $(OUTPUT)%.o: ../../lib/%.c FORCE
 	$(call rule_mkdir)
-	$(call if_changed_dep,cc_o_c)
+	$(call if_changed_dep,host_cc_o_c)
diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
index daed388aa5d7..de513fd08535 100644
--- a/tools/bpf/resolve_btfids/Makefile
+++ b/tools/bpf/resolve_btfids/Makefile
@@ -22,6 +22,9 @@ HOST_OVERRIDES := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" ARCH="$(HOSTARCH)
 		  EXTRA_CFLAGS="$(HOSTCFLAGS) $(KBUILD_HOSTCFLAGS)"
 
 RM      ?= rm
+HOSTCC  ?= gcc
+HOSTLD  ?= ld
+HOSTAR  ?= ar
 CROSS_COMPILE =
 
 OUTPUT ?= $(srctree)/tools/bpf/resolve_btfids/
@@ -64,7 +67,7 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
 LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
 LIBELF_LIBS  := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
 
-CFLAGS += -g \
+HOSTCFLAGS += -g \
           -I$(srctree)/tools/include \
           -I$(srctree)/tools/include/uapi \
           -I$(LIBBPF_INCLUDE) \
@@ -73,7 +76,7 @@ CFLAGS += -g \
 
 LIBS = $(LIBELF_LIBS) -lz
 
-export srctree OUTPUT CFLAGS Q
+export srctree OUTPUT HOSTCFLAGS Q HOSTCC HOSTLD HOSTAR
 include $(srctree)/tools/build/Makefile.include
 
 $(BINARY_IN): fixdep FORCE prepare | $(OUTPUT)

^ permalink raw reply related	[flat|nested] 9+ messages in thread
* Re: [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing
  2023-02-01 10:43   ` Jiri Olsa
@ 2023-02-01 18:57     ` Ian Rogers
  2023-02-01 20:50       ` Jiri Olsa
  0 siblings, 1 reply; 9+ messages in thread
From: Ian Rogers @ 2023-02-01 18:57 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
	KP Singh, Stanislav Fomichev, Hao Luo, Connor OBrien,
	Nathan Chancellor, Kumar Kartikeya Dwivedi, bpf, linux-kernel

On Wed, Feb 1, 2023 at 2:43 AM Jiri Olsa <olsajiri@gmail.com> wrote:
>
> On Wed, Feb 01, 2023 at 11:00:02AM +0100, Jiri Olsa wrote:
> > On Tue, Jan 31, 2023 at 05:50:15PM -0800, Ian Rogers wrote:
> > > Avoid passing CROSS_COMPILE to submakes and ensure CFLAGS is forced to
> > > HOSTCFLAGS for submake builds. This fixes problems with cross
> > > compilation.
> > >
> > > Tidy to not unnecessarily modify/export CFLAGS, make the override for
> > > prepare and build clearer.
> > >
> > > Fixes: 13e07691a16f ("tools/resolve_btfids: Alter how HOSTCC is forced")
> > > Reported-by: Nathan Chancellor <nathan@kernel.org>
> > > Signed-off-by: Ian Rogers <irogers@google.com>
> >
> > hum, that seems to build just the fixdep and skip the resolve_btfids binary
> >
> > make[1]: Entering directory '/home/jolsa/kernel/linux-qemu/build'
> >   GEN     Makefile
> >   CALL    ../scripts/checksyscalls.sh
> >   DESCEND bpf/resolve_btfids
> >   HOSTCC  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep.o
> >   HOSTLD  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep-in.o
> >   LINK    /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep
> >   UPD     include/generated/utsversion.h
> >   CC      init/version-timestamp.o
> >   LD      .tmp_vmlinux.btf
> >   BTF     .btf.vmlinux.bin.o
> > die__process_unit: DW_TAG_label (0xa) @ <0x4f0d4> not handled!
> >
> >   ...
> >
> > die__process_unit: DW_TAG_label (0xa) @ <0xaf91cc3> not handled!
> > die__process_unit: DW_TAG_label (0xa) @ <0xb032fa7> not handled!
> >   LD      .tmp_vmlinux.kallsyms1
> >   NM      .tmp_vmlinux.kallsyms1.syms
> >   KSYMS   .tmp_vmlinux.kallsyms1.S
> >   AS      .tmp_vmlinux.kallsyms1.S
> >   LD      .tmp_vmlinux.kallsyms2
> >   NM      .tmp_vmlinux.kallsyms2.syms
> >   KSYMS   .tmp_vmlinux.kallsyms2.S
> >   AS      .tmp_vmlinux.kallsyms2.S
> >   LD      .tmp_vmlinux.kallsyms3
> >   NM      .tmp_vmlinux.kallsyms3.syms
> >   KSYMS   .tmp_vmlinux.kallsyms3.S
> >   AS      .tmp_vmlinux.kallsyms3.S
> >   LD      vmlinux
> >   BTFIDS  vmlinux
> > ../scripts/link-vmlinux.sh: line 277: ./tools/bpf/resolve_btfids/resolve_btfids: No such file or directory
> > make[2]: *** [../scripts/Makefile.vmlinux:35: vmlinux] Error 127
> > make[2]: *** Deleting file 'vmlinux'
> > make[1]: *** [/home/jolsa/kernel/linux-qemu/Makefile:1264: vmlinux] Error 2
> > make[1]: Leaving directory '/home/jolsa/kernel/linux-qemu/build'
> > make: *** [Makefile:242: __sub-make] Error 2
> >
> > we actually have the hostprogs support in tools/build and we use it for
> > fixdep, I think we should use it also here, I'll check
>
> it doesn't look that bad.. the change below fixes the build for me,
> perhaps we should do that for all the host tools
>
> jirka

I don't mind this. The fixdep vs all thing is just cause by the
ordering in the Makefile, you can fix by specifying the target or add
this patch:
```
--- a/tools/bpf/resolve_btfids/Makefile
+++ b/tools/bpf/resolve_btfids/Makefile
@@ -58,11 +58,11 @@ HOST_OVERRIDES_BUILD := $(HOST_OVERRIDES_PREPARE) \

LIBS = $(LIBELF_LIBS) -lz

+all: $(BINARY)
+
export srctree OUTPUT Q
include $(srctree)/tools/build/Makefile.include

-all: $(BINARY)
-
prepare: $(BPFOBJ) $(SUBCMDOBJ)

$(OUTPUT) $(OUTPUT)/libsubcmd $(LIBBPF_OUT):
```

Should we do this and the hostprogs migration as a follow up? There
isn't that much use of hostprogs in tools, but I like that your change
will show HOSTCC rather than CC during compilation. If we use
hostprogs can we just avoid the overrides altogether?

Thanks,
Ian

>
> ---
> diff --git a/tools/bpf/resolve_btfids/Build b/tools/bpf/resolve_btfids/Build
> index ae82da03f9bf..077de3829c72 100644
> --- a/tools/bpf/resolve_btfids/Build
> +++ b/tools/bpf/resolve_btfids/Build
> @@ -1,3 +1,5 @@
> +hostprogs := resolve_btfids
> +
>  resolve_btfids-y += main.o
>  resolve_btfids-y += rbtree.o
>  resolve_btfids-y += zalloc.o
> @@ -7,4 +9,4 @@ resolve_btfids-y += str_error_r.o
>
>  $(OUTPUT)%.o: ../../lib/%.c FORCE
>         $(call rule_mkdir)
> -       $(call if_changed_dep,cc_o_c)
> +       $(call if_changed_dep,host_cc_o_c)
> diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
> index daed388aa5d7..de513fd08535 100644
> --- a/tools/bpf/resolve_btfids/Makefile
> +++ b/tools/bpf/resolve_btfids/Makefile
> @@ -22,6 +22,9 @@ HOST_OVERRIDES := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" ARCH="$(HOSTARCH)
>                   EXTRA_CFLAGS="$(HOSTCFLAGS) $(KBUILD_HOSTCFLAGS)"
>
>  RM      ?= rm
> +HOSTCC  ?= gcc
> +HOSTLD  ?= ld
> +HOSTAR  ?= ar
>  CROSS_COMPILE =
>
>  OUTPUT ?= $(srctree)/tools/bpf/resolve_btfids/
> @@ -64,7 +67,7 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
>  LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
>  LIBELF_LIBS  := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
>
> -CFLAGS += -g \
> +HOSTCFLAGS += -g \
>            -I$(srctree)/tools/include \
>            -I$(srctree)/tools/include/uapi \
>            -I$(LIBBPF_INCLUDE) \
> @@ -73,7 +76,7 @@ CFLAGS += -g \
>
>  LIBS = $(LIBELF_LIBS) -lz
>
> -export srctree OUTPUT CFLAGS Q
> +export srctree OUTPUT HOSTCFLAGS Q HOSTCC HOSTLD HOSTAR
>  include $(srctree)/tools/build/Makefile.include
>
>  $(BINARY_IN): fixdep FORCE prepare | $(OUTPUT)

^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing
  2023-02-01 18:57     ` Ian Rogers
@ 2023-02-01 20:50       ` Jiri Olsa
  2023-02-01 21:21         ` Nathan Chancellor
  0 siblings, 1 reply; 9+ messages in thread
From: Jiri Olsa @ 2023-02-01 20:50 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Jiri Olsa, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
	KP Singh, Stanislav Fomichev, Hao Luo, Connor OBrien,
	Nathan Chancellor, Kumar Kartikeya Dwivedi, bpf, linux-kernel

On Wed, Feb 01, 2023 at 10:57:40AM -0800, Ian Rogers wrote:
> On Wed, Feb 1, 2023 at 2:43 AM Jiri Olsa <olsajiri@gmail.com> wrote:
> >
> > On Wed, Feb 01, 2023 at 11:00:02AM +0100, Jiri Olsa wrote:
> > > On Tue, Jan 31, 2023 at 05:50:15PM -0800, Ian Rogers wrote:
> > > > Avoid passing CROSS_COMPILE to submakes and ensure CFLAGS is forced to
> > > > HOSTCFLAGS for submake builds. This fixes problems with cross
> > > > compilation.
> > > >
> > > > Tidy to not unnecessarily modify/export CFLAGS, make the override for
> > > > prepare and build clearer.
> > > >
> > > > Fixes: 13e07691a16f ("tools/resolve_btfids: Alter how HOSTCC is forced")
> > > > Reported-by: Nathan Chancellor <nathan@kernel.org>
> > > > Signed-off-by: Ian Rogers <irogers@google.com>
> > >
> > > hum, that seems to build just the fixdep and skip the resolve_btfids binary
> > >
> > > make[1]: Entering directory '/home/jolsa/kernel/linux-qemu/build'
> > >   GEN     Makefile
> > >   CALL    ../scripts/checksyscalls.sh
> > >   DESCEND bpf/resolve_btfids
> > >   HOSTCC  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep.o
> > >   HOSTLD  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep-in.o
> > >   LINK    /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep
> > >   UPD     include/generated/utsversion.h
> > >   CC      init/version-timestamp.o
> > >   LD      .tmp_vmlinux.btf
> > >   BTF     .btf.vmlinux.bin.o
> > > die__process_unit: DW_TAG_label (0xa) @ <0x4f0d4> not handled!
> > >
> > >   ...
> > >
> > > die__process_unit: DW_TAG_label (0xa) @ <0xaf91cc3> not handled!
> > > die__process_unit: DW_TAG_label (0xa) @ <0xb032fa7> not handled!
> > >   LD      .tmp_vmlinux.kallsyms1
> > >   NM      .tmp_vmlinux.kallsyms1.syms
> > >   KSYMS   .tmp_vmlinux.kallsyms1.S
> > >   AS      .tmp_vmlinux.kallsyms1.S
> > >   LD      .tmp_vmlinux.kallsyms2
> > >   NM      .tmp_vmlinux.kallsyms2.syms
> > >   KSYMS   .tmp_vmlinux.kallsyms2.S
> > >   AS      .tmp_vmlinux.kallsyms2.S
> > >   LD      .tmp_vmlinux.kallsyms3
> > >   NM      .tmp_vmlinux.kallsyms3.syms
> > >   KSYMS   .tmp_vmlinux.kallsyms3.S
> > >   AS      .tmp_vmlinux.kallsyms3.S
> > >   LD      vmlinux
> > >   BTFIDS  vmlinux
> > > ../scripts/link-vmlinux.sh: line 277: ./tools/bpf/resolve_btfids/resolve_btfids: No such file or directory
> > > make[2]: *** [../scripts/Makefile.vmlinux:35: vmlinux] Error 127
> > > make[2]: *** Deleting file 'vmlinux'
> > > make[1]: *** [/home/jolsa/kernel/linux-qemu/Makefile:1264: vmlinux] Error 2
> > > make[1]: Leaving directory '/home/jolsa/kernel/linux-qemu/build'
> > > make: *** [Makefile:242: __sub-make] Error 2
> > >
> > > we actually have the hostprogs support in tools/build and we use it for
> > > fixdep, I think we should use it also here, I'll check
> >
> > it doesn't look that bad.. the change below fixes the build for me,
> > perhaps we should do that for all the host tools
> >
> > jirka
> 
> I don't mind this. The fixdep vs all thing is just cause by the
> ordering in the Makefile, you can fix by specifying the target or add
> this patch:
> ```
> --- a/tools/bpf/resolve_btfids/Makefile
> +++ b/tools/bpf/resolve_btfids/Makefile
> @@ -58,11 +58,11 @@ HOST_OVERRIDES_BUILD := $(HOST_OVERRIDES_PREPARE) \
> 
> LIBS = $(LIBELF_LIBS) -lz
> 
> +all: $(BINARY)
> +
> export srctree OUTPUT Q
> include $(srctree)/tools/build/Makefile.include
> 
> -all: $(BINARY)
> -
> prepare: $(BPFOBJ) $(SUBCMDOBJ)
> 
> $(OUTPUT) $(OUTPUT)/libsubcmd $(LIBBPF_OUT):
> ```
> 
> Should we do this and the hostprogs migration as a follow up? There
> isn't that much use of hostprogs in tools, but I like that your change
> will show HOSTCC rather than CC during compilation. If we use
> hostprogs can we just avoid the overrides altogether?

right, I think so.. we can now remove it for the BINARY target and
then for the rest of 'prepare' once they are converted to hostprogs..
not sure how 'hostprogs' will work for library, will need to check

I'll try to send the format patch with the fix below tomorrow

Nathan, any chance you could test it?

thanks,
jirka

> 
> Thanks,
> Ian
> 
> >
> > ---
> > diff --git a/tools/bpf/resolve_btfids/Build b/tools/bpf/resolve_btfids/Build
> > index ae82da03f9bf..077de3829c72 100644
> > --- a/tools/bpf/resolve_btfids/Build
> > +++ b/tools/bpf/resolve_btfids/Build
> > @@ -1,3 +1,5 @@
> > +hostprogs := resolve_btfids
> > +
> >  resolve_btfids-y += main.o
> >  resolve_btfids-y += rbtree.o
> >  resolve_btfids-y += zalloc.o
> > @@ -7,4 +9,4 @@ resolve_btfids-y += str_error_r.o
> >
> >  $(OUTPUT)%.o: ../../lib/%.c FORCE
> >         $(call rule_mkdir)
> > -       $(call if_changed_dep,cc_o_c)
> > +       $(call if_changed_dep,host_cc_o_c)
> > diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
> > index daed388aa5d7..de513fd08535 100644
> > --- a/tools/bpf/resolve_btfids/Makefile
> > +++ b/tools/bpf/resolve_btfids/Makefile
> > @@ -22,6 +22,9 @@ HOST_OVERRIDES := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" ARCH="$(HOSTARCH)
> >                   EXTRA_CFLAGS="$(HOSTCFLAGS) $(KBUILD_HOSTCFLAGS)"
> >
> >  RM      ?= rm
> > +HOSTCC  ?= gcc
> > +HOSTLD  ?= ld
> > +HOSTAR  ?= ar
> >  CROSS_COMPILE =
> >
> >  OUTPUT ?= $(srctree)/tools/bpf/resolve_btfids/
> > @@ -64,7 +67,7 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
> >  LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
> >  LIBELF_LIBS  := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
> >
> > -CFLAGS += -g \
> > +HOSTCFLAGS += -g \
> >            -I$(srctree)/tools/include \
> >            -I$(srctree)/tools/include/uapi \
> >            -I$(LIBBPF_INCLUDE) \
> > @@ -73,7 +76,7 @@ CFLAGS += -g \
> >
> >  LIBS = $(LIBELF_LIBS) -lz
> >
> > -export srctree OUTPUT CFLAGS Q
> > +export srctree OUTPUT HOSTCFLAGS Q HOSTCC HOSTLD HOSTAR
> >  include $(srctree)/tools/build/Makefile.include
> >
> >  $(BINARY_IN): fixdep FORCE prepare | $(OUTPUT)

^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing
  2023-02-01 20:50       ` Jiri Olsa
@ 2023-02-01 21:21         ` Nathan Chancellor
  2023-02-01 21:42           ` Ian Rogers
  2023-02-02 11:32           ` Jiri Olsa
  0 siblings, 2 replies; 9+ messages in thread
From: Nathan Chancellor @ 2023-02-01 21:21 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Ian Rogers, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
	KP Singh, Stanislav Fomichev, Hao Luo, Connor OBrien,
	Kumar Kartikeya Dwivedi, bpf, linux-kernel

On Wed, Feb 01, 2023 at 09:50:38PM +0100, Jiri Olsa wrote:
> On Wed, Feb 01, 2023 at 10:57:40AM -0800, Ian Rogers wrote:
> > On Wed, Feb 1, 2023 at 2:43 AM Jiri Olsa <olsajiri@gmail.com> wrote:
> > >
> > > On Wed, Feb 01, 2023 at 11:00:02AM +0100, Jiri Olsa wrote:
> > > > On Tue, Jan 31, 2023 at 05:50:15PM -0800, Ian Rogers wrote:
> > > > > Avoid passing CROSS_COMPILE to submakes and ensure CFLAGS is forced to
> > > > > HOSTCFLAGS for submake builds. This fixes problems with cross
> > > > > compilation.
> > > > >
> > > > > Tidy to not unnecessarily modify/export CFLAGS, make the override for
> > > > > prepare and build clearer.
> > > > >
> > > > > Fixes: 13e07691a16f ("tools/resolve_btfids: Alter how HOSTCC is forced")
> > > > > Reported-by: Nathan Chancellor <nathan@kernel.org>
> > > > > Signed-off-by: Ian Rogers <irogers@google.com>
> > > >
> > > > hum, that seems to build just the fixdep and skip the resolve_btfids binary
> > > >
> > > > make[1]: Entering directory '/home/jolsa/kernel/linux-qemu/build'
> > > >   GEN     Makefile
> > > >   CALL    ../scripts/checksyscalls.sh
> > > >   DESCEND bpf/resolve_btfids
> > > >   HOSTCC  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep.o
> > > >   HOSTLD  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep-in.o
> > > >   LINK    /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep
> > > >   UPD     include/generated/utsversion.h
> > > >   CC      init/version-timestamp.o
> > > >   LD      .tmp_vmlinux.btf
> > > >   BTF     .btf.vmlinux.bin.o
> > > > die__process_unit: DW_TAG_label (0xa) @ <0x4f0d4> not handled!
> > > >
> > > >   ...
> > > >
> > > > die__process_unit: DW_TAG_label (0xa) @ <0xaf91cc3> not handled!
> > > > die__process_unit: DW_TAG_label (0xa) @ <0xb032fa7> not handled!
> > > >   LD      .tmp_vmlinux.kallsyms1
> > > >   NM      .tmp_vmlinux.kallsyms1.syms
> > > >   KSYMS   .tmp_vmlinux.kallsyms1.S
> > > >   AS      .tmp_vmlinux.kallsyms1.S
> > > >   LD      .tmp_vmlinux.kallsyms2
> > > >   NM      .tmp_vmlinux.kallsyms2.syms
> > > >   KSYMS   .tmp_vmlinux.kallsyms2.S
> > > >   AS      .tmp_vmlinux.kallsyms2.S
> > > >   LD      .tmp_vmlinux.kallsyms3
> > > >   NM      .tmp_vmlinux.kallsyms3.syms
> > > >   KSYMS   .tmp_vmlinux.kallsyms3.S
> > > >   AS      .tmp_vmlinux.kallsyms3.S
> > > >   LD      vmlinux
> > > >   BTFIDS  vmlinux
> > > > ../scripts/link-vmlinux.sh: line 277: ./tools/bpf/resolve_btfids/resolve_btfids: No such file or directory
> > > > make[2]: *** [../scripts/Makefile.vmlinux:35: vmlinux] Error 127
> > > > make[2]: *** Deleting file 'vmlinux'
> > > > make[1]: *** [/home/jolsa/kernel/linux-qemu/Makefile:1264: vmlinux] Error 2
> > > > make[1]: Leaving directory '/home/jolsa/kernel/linux-qemu/build'
> > > > make: *** [Makefile:242: __sub-make] Error 2
> > > >
> > > > we actually have the hostprogs support in tools/build and we use it for
> > > > fixdep, I think we should use it also here, I'll check
> > >
> > > it doesn't look that bad.. the change below fixes the build for me,
> > > perhaps we should do that for all the host tools
> > >
> > > jirka
> > 
> > I don't mind this. The fixdep vs all thing is just cause by the
> > ordering in the Makefile, you can fix by specifying the target or add
> > this patch:
> > ```
> > --- a/tools/bpf/resolve_btfids/Makefile
> > +++ b/tools/bpf/resolve_btfids/Makefile
> > @@ -58,11 +58,11 @@ HOST_OVERRIDES_BUILD := $(HOST_OVERRIDES_PREPARE) \
> > 
> > LIBS = $(LIBELF_LIBS) -lz
> > 
> > +all: $(BINARY)
> > +
> > export srctree OUTPUT Q
> > include $(srctree)/tools/build/Makefile.include
> > 
> > -all: $(BINARY)
> > -
> > prepare: $(BPFOBJ) $(SUBCMDOBJ)
> > 
> > $(OUTPUT) $(OUTPUT)/libsubcmd $(LIBBPF_OUT):
> > ```
> > 
> > Should we do this and the hostprogs migration as a follow up? There
> > isn't that much use of hostprogs in tools, but I like that your change
> > will show HOSTCC rather than CC during compilation. If we use
> > hostprogs can we just avoid the overrides altogether?
> 
> right, I think so.. we can now remove it for the BINARY target and
> then for the rest of 'prepare' once they are converted to hostprogs..
> not sure how 'hostprogs' will work for library, will need to check
> 
> I'll try to send the format patch with the fix below tomorrow
> 
> Nathan, any chance you could test it?


Sure thing, would you happen to have a single patch file of what I
should test? I am a little confused from reading the thread (unless what
you want tested has not been sent yet).

Cheers,
Nathan

> > > ---
> > > diff --git a/tools/bpf/resolve_btfids/Build b/tools/bpf/resolve_btfids/Build
> > > index ae82da03f9bf..077de3829c72 100644
> > > --- a/tools/bpf/resolve_btfids/Build
> > > +++ b/tools/bpf/resolve_btfids/Build
> > > @@ -1,3 +1,5 @@
> > > +hostprogs := resolve_btfids
> > > +
> > >  resolve_btfids-y += main.o
> > >  resolve_btfids-y += rbtree.o
> > >  resolve_btfids-y += zalloc.o
> > > @@ -7,4 +9,4 @@ resolve_btfids-y += str_error_r.o
> > >
> > >  $(OUTPUT)%.o: ../../lib/%.c FORCE
> > >         $(call rule_mkdir)
> > > -       $(call if_changed_dep,cc_o_c)
> > > +       $(call if_changed_dep,host_cc_o_c)
> > > diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
> > > index daed388aa5d7..de513fd08535 100644
> > > --- a/tools/bpf/resolve_btfids/Makefile
> > > +++ b/tools/bpf/resolve_btfids/Makefile
> > > @@ -22,6 +22,9 @@ HOST_OVERRIDES := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" ARCH="$(HOSTARCH)
> > >                   EXTRA_CFLAGS="$(HOSTCFLAGS) $(KBUILD_HOSTCFLAGS)"
> > >
> > >  RM      ?= rm
> > > +HOSTCC  ?= gcc
> > > +HOSTLD  ?= ld
> > > +HOSTAR  ?= ar
> > >  CROSS_COMPILE =
> > >
> > >  OUTPUT ?= $(srctree)/tools/bpf/resolve_btfids/
> > > @@ -64,7 +67,7 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
> > >  LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
> > >  LIBELF_LIBS  := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
> > >
> > > -CFLAGS += -g \
> > > +HOSTCFLAGS += -g \
> > >            -I$(srctree)/tools/include \
> > >            -I$(srctree)/tools/include/uapi \
> > >            -I$(LIBBPF_INCLUDE) \
> > > @@ -73,7 +76,7 @@ CFLAGS += -g \
> > >
> > >  LIBS = $(LIBELF_LIBS) -lz
> > >
> > > -export srctree OUTPUT CFLAGS Q
> > > +export srctree OUTPUT HOSTCFLAGS Q HOSTCC HOSTLD HOSTAR
> > >  include $(srctree)/tools/build/Makefile.include
> > >
> > >  $(BINARY_IN): fixdep FORCE prepare | $(OUTPUT)

^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing
  2023-02-01 21:21         ` Nathan Chancellor
@ 2023-02-01 21:42           ` Ian Rogers
  2023-02-02 11:32           ` Jiri Olsa
  1 sibling, 0 replies; 9+ messages in thread
From: Ian Rogers @ 2023-02-01 21:42 UTC (permalink / raw)
  To: Nathan Chancellor
  Cc: Jiri Olsa, Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
	Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
	KP Singh, Stanislav Fomichev, Hao Luo, Connor OBrien,
	Kumar Kartikeya Dwivedi, bpf, linux-kernel

On Wed, Feb 1, 2023 at 1:21 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Wed, Feb 01, 2023 at 09:50:38PM +0100, Jiri Olsa wrote:
> > On Wed, Feb 01, 2023 at 10:57:40AM -0800, Ian Rogers wrote:
> > > On Wed, Feb 1, 2023 at 2:43 AM Jiri Olsa <olsajiri@gmail.com> wrote:
> > > >
> > > > On Wed, Feb 01, 2023 at 11:00:02AM +0100, Jiri Olsa wrote:
> > > > > On Tue, Jan 31, 2023 at 05:50:15PM -0800, Ian Rogers wrote:
> > > > > > Avoid passing CROSS_COMPILE to submakes and ensure CFLAGS is forced to
> > > > > > HOSTCFLAGS for submake builds. This fixes problems with cross
> > > > > > compilation.
> > > > > >
> > > > > > Tidy to not unnecessarily modify/export CFLAGS, make the override for
> > > > > > prepare and build clearer.
> > > > > >
> > > > > > Fixes: 13e07691a16f ("tools/resolve_btfids: Alter how HOSTCC is forced")
> > > > > > Reported-by: Nathan Chancellor <nathan@kernel.org>
> > > > > > Signed-off-by: Ian Rogers <irogers@google.com>
> > > > >
> > > > > hum, that seems to build just the fixdep and skip the resolve_btfids binary
> > > > >
> > > > > make[1]: Entering directory '/home/jolsa/kernel/linux-qemu/build'
> > > > >   GEN     Makefile
> > > > >   CALL    ../scripts/checksyscalls.sh
> > > > >   DESCEND bpf/resolve_btfids
> > > > >   HOSTCC  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep.o
> > > > >   HOSTLD  /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep-in.o
> > > > >   LINK    /home/jolsa/kernel/linux-qemu/build/tools/bpf/resolve_btfids/fixdep
> > > > >   UPD     include/generated/utsversion.h
> > > > >   CC      init/version-timestamp.o
> > > > >   LD      .tmp_vmlinux.btf
> > > > >   BTF     .btf.vmlinux.bin.o
> > > > > die__process_unit: DW_TAG_label (0xa) @ <0x4f0d4> not handled!
> > > > >
> > > > >   ...
> > > > >
> > > > > die__process_unit: DW_TAG_label (0xa) @ <0xaf91cc3> not handled!
> > > > > die__process_unit: DW_TAG_label (0xa) @ <0xb032fa7> not handled!
> > > > >   LD      .tmp_vmlinux.kallsyms1
> > > > >   NM      .tmp_vmlinux.kallsyms1.syms
> > > > >   KSYMS   .tmp_vmlinux.kallsyms1.S
> > > > >   AS      .tmp_vmlinux.kallsyms1.S
> > > > >   LD      .tmp_vmlinux.kallsyms2
> > > > >   NM      .tmp_vmlinux.kallsyms2.syms
> > > > >   KSYMS   .tmp_vmlinux.kallsyms2.S
> > > > >   AS      .tmp_vmlinux.kallsyms2.S
> > > > >   LD      .tmp_vmlinux.kallsyms3
> > > > >   NM      .tmp_vmlinux.kallsyms3.syms
> > > > >   KSYMS   .tmp_vmlinux.kallsyms3.S
> > > > >   AS      .tmp_vmlinux.kallsyms3.S
> > > > >   LD      vmlinux
> > > > >   BTFIDS  vmlinux
> > > > > ../scripts/link-vmlinux.sh: line 277: ./tools/bpf/resolve_btfids/resolve_btfids: No such file or directory
> > > > > make[2]: *** [../scripts/Makefile.vmlinux:35: vmlinux] Error 127
> > > > > make[2]: *** Deleting file 'vmlinux'
> > > > > make[1]: *** [/home/jolsa/kernel/linux-qemu/Makefile:1264: vmlinux] Error 2
> > > > > make[1]: Leaving directory '/home/jolsa/kernel/linux-qemu/build'
> > > > > make: *** [Makefile:242: __sub-make] Error 2
> > > > >
> > > > > we actually have the hostprogs support in tools/build and we use it for
> > > > > fixdep, I think we should use it also here, I'll check
> > > >
> > > > it doesn't look that bad.. the change below fixes the build for me,
> > > > perhaps we should do that for all the host tools
> > > >
> > > > jirka
> > >
> > > I don't mind this. The fixdep vs all thing is just cause by the
> > > ordering in the Makefile, you can fix by specifying the target or add
> > > this patch:
> > > ```
> > > --- a/tools/bpf/resolve_btfids/Makefile
> > > +++ b/tools/bpf/resolve_btfids/Makefile
> > > @@ -58,11 +58,11 @@ HOST_OVERRIDES_BUILD := $(HOST_OVERRIDES_PREPARE) \
> > >
> > > LIBS = $(LIBELF_LIBS) -lz
> > >
> > > +all: $(BINARY)
> > > +
> > > export srctree OUTPUT Q
> > > include $(srctree)/tools/build/Makefile.include
> > >
> > > -all: $(BINARY)
> > > -
> > > prepare: $(BPFOBJ) $(SUBCMDOBJ)
> > >
> > > $(OUTPUT) $(OUTPUT)/libsubcmd $(LIBBPF_OUT):
> > > ```
> > >
> > > Should we do this and the hostprogs migration as a follow up? There
> > > isn't that much use of hostprogs in tools, but I like that your change
> > > will show HOSTCC rather than CC during compilation. If we use
> > > hostprogs can we just avoid the overrides altogether?
> >
> > right, I think so.. we can now remove it for the BINARY target and
> > then for the rest of 'prepare' once they are converted to hostprogs..
> > not sure how 'hostprogs' will work for library, will need to check
> >
> > I'll try to send the format patch with the fix below tomorrow
> >
> > Nathan, any chance you could test it?
>
>
> Sure thing, would you happen to have a single patch file of what I
> should test? I am a little confused from reading the thread (unless what
> you want tested has not been sent yet).
>
> Cheers,
> Nathan

I resent the change with the 'all' target moved first:
https://lore.kernel.org/lkml/20230201213743.44674-1-irogers@google.com/
but I think Jiri's fix is better.

Thanks,
Ian

> > > > ---
> > > > diff --git a/tools/bpf/resolve_btfids/Build b/tools/bpf/resolve_btfids/Build
> > > > index ae82da03f9bf..077de3829c72 100644
> > > > --- a/tools/bpf/resolve_btfids/Build
> > > > +++ b/tools/bpf/resolve_btfids/Build
> > > > @@ -1,3 +1,5 @@
> > > > +hostprogs := resolve_btfids
> > > > +
> > > >  resolve_btfids-y += main.o
> > > >  resolve_btfids-y += rbtree.o
> > > >  resolve_btfids-y += zalloc.o
> > > > @@ -7,4 +9,4 @@ resolve_btfids-y += str_error_r.o
> > > >
> > > >  $(OUTPUT)%.o: ../../lib/%.c FORCE
> > > >         $(call rule_mkdir)
> > > > -       $(call if_changed_dep,cc_o_c)
> > > > +       $(call if_changed_dep,host_cc_o_c)
> > > > diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
> > > > index daed388aa5d7..de513fd08535 100644
> > > > --- a/tools/bpf/resolve_btfids/Makefile
> > > > +++ b/tools/bpf/resolve_btfids/Makefile
> > > > @@ -22,6 +22,9 @@ HOST_OVERRIDES := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" ARCH="$(HOSTARCH)
> > > >                   EXTRA_CFLAGS="$(HOSTCFLAGS) $(KBUILD_HOSTCFLAGS)"
> > > >
> > > >  RM      ?= rm
> > > > +HOSTCC  ?= gcc
> > > > +HOSTLD  ?= ld
> > > > +HOSTAR  ?= ar
> > > >  CROSS_COMPILE =
> > > >
> > > >  OUTPUT ?= $(srctree)/tools/bpf/resolve_btfids/
> > > > @@ -64,7 +67,7 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
> > > >  LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
> > > >  LIBELF_LIBS  := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
> > > >
> > > > -CFLAGS += -g \
> > > > +HOSTCFLAGS += -g \
> > > >            -I$(srctree)/tools/include \
> > > >            -I$(srctree)/tools/include/uapi \
> > > >            -I$(LIBBPF_INCLUDE) \
> > > > @@ -73,7 +76,7 @@ CFLAGS += -g \
> > > >
> > > >  LIBS = $(LIBELF_LIBS) -lz
> > > >
> > > > -export srctree OUTPUT CFLAGS Q
> > > > +export srctree OUTPUT HOSTCFLAGS Q HOSTCC HOSTLD HOSTAR
> > > >  include $(srctree)/tools/build/Makefile.include
> > > >
> > > >  $(BINARY_IN): fixdep FORCE prepare | $(OUTPUT)

^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing
  2023-02-01 21:21         ` Nathan Chancellor
  2023-02-01 21:42           ` Ian Rogers
@ 2023-02-02 11:32           ` Jiri Olsa
  2023-02-03 18:41             ` Vladimir Oltean
  1 sibling, 1 reply; 9+ messages in thread
From: Jiri Olsa @ 2023-02-02 11:32 UTC (permalink / raw)
  To: Nathan Chancellor
  Cc: Jiri Olsa, Ian Rogers, Alexei Starovoitov, Daniel Borkmann,
	Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
	John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo,
	Connor OBrien, Kumar Kartikeya Dwivedi, bpf, linux-kernel

On Wed, Feb 01, 2023 at 02:21:16PM -0700, Nathan Chancellor wrote:

SNIP

> > > Should we do this and the hostprogs migration as a follow up? There
> > > isn't that much use of hostprogs in tools, but I like that your change
> > > will show HOSTCC rather than CC during compilation. If we use
> > > hostprogs can we just avoid the overrides altogether?
> > 
> > right, I think so.. we can now remove it for the BINARY target and
> > then for the rest of 'prepare' once they are converted to hostprogs..
> > not sure how 'hostprogs' will work for library, will need to check
> > 
> > I'll try to send the format patch with the fix below tomorrow
> > 
> > Nathan, any chance you could test it?
> 
> 
> Sure thing, would you happen to have a single patch file of what I
> should test? I am a little confused from reading the thread (unless what
> you want tested has not been sent yet).

I meant the change below, but I posted the formal patch now:
  https://lore.kernel.org/bpf/20230202112839.1131892-1-jolsa@kernel.org/T/#u

thanks,
jirka

> 
> Cheers,
> Nathan
> 
> > > > ---
> > > > diff --git a/tools/bpf/resolve_btfids/Build b/tools/bpf/resolve_btfids/Build
> > > > index ae82da03f9bf..077de3829c72 100644
> > > > --- a/tools/bpf/resolve_btfids/Build
> > > > +++ b/tools/bpf/resolve_btfids/Build
> > > > @@ -1,3 +1,5 @@
> > > > +hostprogs := resolve_btfids
> > > > +
> > > >  resolve_btfids-y += main.o
> > > >  resolve_btfids-y += rbtree.o
> > > >  resolve_btfids-y += zalloc.o
> > > > @@ -7,4 +9,4 @@ resolve_btfids-y += str_error_r.o
> > > >
> > > >  $(OUTPUT)%.o: ../../lib/%.c FORCE
> > > >         $(call rule_mkdir)
> > > > -       $(call if_changed_dep,cc_o_c)
> > > > +       $(call if_changed_dep,host_cc_o_c)
> > > > diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
> > > > index daed388aa5d7..de513fd08535 100644
> > > > --- a/tools/bpf/resolve_btfids/Makefile
> > > > +++ b/tools/bpf/resolve_btfids/Makefile
> > > > @@ -22,6 +22,9 @@ HOST_OVERRIDES := AR="$(HOSTAR)" CC="$(HOSTCC)" LD="$(HOSTLD)" ARCH="$(HOSTARCH)
> > > >                   EXTRA_CFLAGS="$(HOSTCFLAGS) $(KBUILD_HOSTCFLAGS)"
> > > >
> > > >  RM      ?= rm
> > > > +HOSTCC  ?= gcc
> > > > +HOSTLD  ?= ld
> > > > +HOSTAR  ?= ar
> > > >  CROSS_COMPILE =
> > > >
> > > >  OUTPUT ?= $(srctree)/tools/bpf/resolve_btfids/
> > > > @@ -64,7 +67,7 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(LIBBPF_OU
> > > >  LIBELF_FLAGS := $(shell $(HOSTPKG_CONFIG) libelf --cflags 2>/dev/null)
> > > >  LIBELF_LIBS  := $(shell $(HOSTPKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
> > > >
> > > > -CFLAGS += -g \
> > > > +HOSTCFLAGS += -g \
> > > >            -I$(srctree)/tools/include \
> > > >            -I$(srctree)/tools/include/uapi \
> > > >            -I$(LIBBPF_INCLUDE) \
> > > > @@ -73,7 +76,7 @@ CFLAGS += -g \
> > > >
> > > >  LIBS = $(LIBELF_LIBS) -lz
> > > >
> > > > -export srctree OUTPUT CFLAGS Q
> > > > +export srctree OUTPUT HOSTCFLAGS Q HOSTCC HOSTLD HOSTAR
> > > >  include $(srctree)/tools/build/Makefile.include
> > > >
> > > >  $(BINARY_IN): fixdep FORCE prepare | $(OUTPUT)

^ permalink raw reply	[flat|nested] 9+ messages in thread
* Re: [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing
  2023-02-02 11:32           ` Jiri Olsa
@ 2023-02-03 18:41             ` Vladimir Oltean
  0 siblings, 0 replies; 9+ messages in thread
From: Vladimir Oltean @ 2023-02-03 18:41 UTC (permalink / raw)
  To: Jiri Olsa, Daniel Borkmann, Alexei Starovoitov
  Cc: Nathan Chancellor, Ian Rogers, Andrii Nakryiko, Martin KaFai Lau,
	Song Liu, Yonghong Song, John Fastabend, KP Singh,
	Stanislav Fomichev, Hao Luo, Connor OBrien,
	Kumar Kartikeya Dwivedi, bpf, linux-kernel

Hi,

On Thu, Feb 02, 2023 at 12:32:33PM +0100, Jiri Olsa wrote:
> On Wed, Feb 01, 2023 at 02:21:16PM -0700, Nathan Chancellor wrote:
> 
> SNIP
> 
> > > > Should we do this and the hostprogs migration as a follow up? There
> > > > isn't that much use of hostprogs in tools, but I like that your change
> > > > will show HOSTCC rather than CC during compilation. If we use
> > > > hostprogs can we just avoid the overrides altogether?
> > > 
> > > right, I think so.. we can now remove it for the BINARY target and
> > > then for the rest of 'prepare' once they are converted to hostprogs..
> > > not sure how 'hostprogs' will work for library, will need to check
> > > 
> > > I'll try to send the format patch with the fix below tomorrow
> > > 
> > > Nathan, any chance you could test it?
> > 
> > 
> > Sure thing, would you happen to have a single patch file of what I
> > should test? I am a little confused from reading the thread (unless what
> > you want tested has not been sent yet).
> 
> I meant the change below, but I posted the formal patch now:
>   https://lore.kernel.org/bpf/20230202112839.1131892-1-jolsa@kernel.org/T/#u

I've been chasing the links posted by Ian, but now I can't comment on patch
"tools/resolve_btfids: Compile resolve_btfids as host program", because
lkml wasn't CCed, and I'm not subscribed to the bpf ML. So I'm commenting here.

Can someone please apply the patch linked above? Compiling with
CONFIG_DEBUG_INFO_BTF=y is also broken on net-next.

Maintainers can add my

Tested-by: Vladimir Oltean <vladimir.oltean@nxp.com>

on that patch if it helps speed up the process.

^ permalink raw reply	[flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-02-03 18:41 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-01  1:50 [PATCH v1] tools/resolve_btfids: Tidy host CFLAGS forcing Ian Rogers
2023-02-01  9:59 ` Jiri Olsa
2023-02-01 10:43   ` Jiri Olsa
2023-02-01 18:57     ` Ian Rogers
2023-02-01 20:50       ` Jiri Olsa
2023-02-01 21:21         ` Nathan Chancellor
2023-02-01 21:42           ` Ian Rogers
2023-02-02 11:32           ` Jiri Olsa
2023-02-03 18:41             ` Vladimir Oltean

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).