Re: [bpf-next PATCH v2 4/4] kbuild: refactor module BTF rule

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Sun, Oct 22, 2023 at 1:24 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Sun, Oct 22, 2023 at 4:33 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Sat, Oct 21, 2023 at 4:38 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > On Sat, Oct 21, 2023 at 5:52 AM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Fri, Oct 20, 2023 at 12:03 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > >
> > > > > On Fri, Oct 20, 2023 at 7:55 AM Andrii Nakryiko
> > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > >
> > > > > > On Thu, Oct 19, 2023 at 1:15 AM Jiri Olsa <olsajiri@gmail.com> wrote:
> > > > > > >
> > > > > > > On Thu, Oct 19, 2023 at 12:19:50AM +0900, Masahiro Yamada wrote:
> > > > > > > > newer_prereqs_except and if_changed_except are ugly hacks of the
> > > > > > > > newer-prereqs and if_changed in scripts/Kbuild.include.
> > > > > > > >
> > > > > > > > Remove.
> > > > > > > >
> > > > > > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > > > > > > > ---
> > > > > > > >
> > > > > > > > Changes in v2:
> > > > > > > >   - Fix if_changed_except to if_changed
> > > > > > > >
> > > > > > > >  scripts/Makefile.modfinal | 25 ++++++-------------------
> > > > > > > >  1 file changed, 6 insertions(+), 19 deletions(-)
> > > > > > > >
> > > > > > > > diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> > > > > > > > index 9fd7a26e4fe9..fc07854bb7b9 100644
> > > > > > > > --- a/scripts/Makefile.modfinal
> > > > > > > > +++ b/scripts/Makefile.modfinal
> > > > > > > > @@ -19,6 +19,9 @@ vmlinux :=
> > > > > > > >  ifdef CONFIG_DEBUG_INFO_BTF_MODULES
> > > > > > > >  ifneq ($(wildcard vmlinux),)
> > > > > > > >  vmlinux := vmlinux
> > > > > > > > +cmd_btf = ; \
> > > > > > > > +     LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) --btf_base vmlinux $@; \
> > > > > > > > +     $(RESOLVE_BTFIDS) -b vmlinux $@
> > > > > > > >  else
> > > > > > > >  $(warning Skipping BTF generation due to unavailability of vmlinux)
> > > > > > > >  endif
> > > > > > > > @@ -41,27 +44,11 @@ quiet_cmd_ld_ko_o = LD [M]  $@
> > > > > > > >        cmd_ld_ko_o +=                                                 \
> > > > > > > >       $(LD) -r $(KBUILD_LDFLAGS)                                      \
> > > > > > > >               $(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE)              \
> > > > > > > > -             -T scripts/module.lds -o $@ $(filter %.o, $^)
> > > > > > > > +             -T scripts/module.lds -o $@ $(filter %.o, $^)           \
> > > > > > > > +     $(cmd_btf)
> > > > > > > >
> > > > > > > > -quiet_cmd_btf_ko = BTF [M] $@
> > > > > > >
> > > > > > > nit not sure it's intentional but we no longer display 'BTF [M] ...ko' lines,
> > > > > > > I don't mind not displaying that, but we should mention that in changelog
> > > > > > >
> > > > > >
> > > > > > Thanks for spotting this! I think those messages are useful and
> > > > > > important to keep. Masahiro, is it possible to preserve them?
> > > > >
> > > > >
> > > > >
> > > > > No, I do not think so.
> > > > >
> > > >
> > > > That's too bad, I think it's a useful one.
> > >
> > >
> > >
> > > I prioritize that the code is correct.
> > >
> >
> > Could you please also prioritize not regressing informativeness of a
> > build log? With your changes it's not clear now if BTF was generated
> > or not for a kernel module, while previously it was obvious and was
> > easy to spot if for some reason BTF was not generated. I'd like to
> > preserve this
> > property, thank you.
> >
> > E.g, can we still have BTF generation as a separate command and do a
> > separate $(call if_changed,btf_ko)? Or something along those lines.
> > Would that work?
>
> If we have an intermediate file (say, *.no-btf.ko),
> it would make sense to have separate
> $(call if_changed,ld_ko_o) and $(call if_changed,btf_ko).

Currently we don't generate intermediate files, but we do rewrite
original .ko file as a post-processing step.

And that rewriting step might not happen depending on Kconfig and
toolchain (e.g., too old pahole makes it impossible to generate kernel
module BTF). And that's why having a separate BTF [M] message in the
build log is important.

>
>
>            LD                 RESOLVE_BTFIDS
>  *.mod.o  ------> *.no-btf.ko ------------> *.ko
>
>
> When vmlinux is changed, only the second step would
> be re-run, but that would require extra file copy.

Today we rewrite .ko with a new .ko ELF file which gains a new ELF
section (.BTF), so we already pay this price when BTF is enabled (if
that's your concern).

>
> Is this what you want to see?

I don't have strong preferences for exact implementation, but what you
propose will work, I think. What I'd like to avoid is unnecessarily
relinking .ko files if all we need to do is regenerate BTF.

>
>
>
>
>
> >
> > >
> > >
> > > >
> > > > > Your code is wrong.
> > > > >
> > > >
> > > > Could be, but note the comment you are removing:
> > > >
> > > > # Re-generate module BTFs if either module's .ko or vmlinux changed
> > > >
> > > > BTF has to be re-generated not just when module .ko is regenerated,
> > > > but also when the vmlinux image itself changes.
> > > >
> > > > I don't see where this is done with your changes. Can you please point
> > > > it out explicitly?
> > >
> > >
> > >
> > > That is too obvious; %.ko depends on $(vmlinux).
> >
> > Thank you for your gracious answer. We used to not rebuild module's
> > .ko's when vmlinux didn't change (but we did regen BTFs), and that's
> > why I was confused. Now we forcefully recompile modules, which is a
> > change in behavior which would be nice to call out in the commit
> > message.
> >
> >
> > >
> > >
> > >
> > > %.ko: %.o %.mod.o scripts/module.lds $(vmlinux) FORCE
> > >
> > >
> > >
> > >
> > > --
> > > Best Regards
> > > Masahiro Yamada
>
>
>
> --
> Best Regards
> Masahiro Yamada