Re: [PATCH v2] scripts: `make rust-analyzer` for out-of-tree modules

[Vinay Varma <varmavinaym@gmail.com>]

Sorry, got caught up with another project and lost track of this thread.
I have updated the patch and replied to some of the threads inline.
On Sat, Jan 21, 2023 at 06:18:01PM +0900, Masahiro Yamada wrote:
> On Sat, Jan 21, 2023 at 2:25 PM Vinay Varma <varmavinaym@gmail.com> wrote:
> >
> > Adds support for out-of-tree rust modules to use the `rust-analyzer`
> > make target to generate the rust-project.json file.
> >
> > The change involves adding an optional parameter `external_src` to the
> > `generate_rust_analyzer.py` which expects the path to the out-of-tree
> > module's source directory. When this parameter is passed, I have chosen
> > not to add the non-core modules (samples and drivers) into the result
> > since these are not expected to be used in third party modules. Related
> > changes are also made to the Makefile and rust/Makefile allowing the
> > `rust-analyzer` target to be used for out-of-tree modules as well.
> >
> > Link: https://github.com/Rust-for-Linux/linux/pull/914
> > Link: https://github.com/Rust-for-Linux/rust-out-of-tree-module/pull/2
> >
> > Signed-off-by: Vinay Varma <varmavinaym@gmail.com>
> > ---
> >  Makefile                          | 12 +++++++-----
> >  rust/Makefile                     |  6 ++++--
> >  scripts/generate_rust_analyzer.py | 31 ++++++++++++++++++-------------
> >  3 files changed, 29 insertions(+), 20 deletions(-)
> >
> > diff --git a/Makefile b/Makefile
> > index f41ec8c8426b..a055a316d2a4 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -1831,11 +1831,6 @@ rustfmt:
> >  rustfmtcheck: rustfmt_flags = --check
> >  rustfmtcheck: rustfmt
> >
> > -# IDE support targets
> > -PHONY += rust-analyzer
> > -rust-analyzer:
> > -       $(Q)$(MAKE) $(build)=rust $@
> > -
> >  # Misc
> >  # ---------------------------------------------------------------------------
> >
> > @@ -1888,6 +1883,7 @@ help:
> >         @echo  '  modules         - default target, build the module(s)'
> >         @echo  '  modules_install - install the module'
> >         @echo  '  clean           - remove generated files in module directory only'
> > +       @echo  '  rust-analyzer   - generate rust-project.json rust-analyzer support file'
> >         @echo  ''
> >
> >  endif # KBUILD_EXTMOD
> > @@ -2022,6 +2018,12 @@ quiet_cmd_tags = GEN     $@
> >  tags TAGS cscope gtags: FORCE
> >         $(call cmd,tags)
> >
> > +# IDE support targets
> > +PHONY += rust-analyzer
> > +rust-analyzer:
> > +       $(Q)$(MAKE) $(build)=rust $@
> > +
> > +
> 
> 
> Extra empty line.
> 
> 
> >  # Script to generate missing namespace dependencies
> >  # ---------------------------------------------------------------------------
> >
> > diff --git a/rust/Makefile b/rust/Makefile
> > index 8f598a904f38..41c1435cd8d4 100644
> > --- a/rust/Makefile
> > +++ b/rust/Makefile
> > @@ -389,8 +389,10 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L
> >         $(if $(rustc_objcopy),;$(OBJCOPY) $(rustc_objcopy) $@)
> >
> >  rust-analyzer:
> > -       $(Q)$(srctree)/scripts/generate_rust_analyzer.py $(srctree) $(objtree) \
> > -               $(RUST_LIB_SRC) > $(objtree)/rust-project.json
> > +       $(Q)$(srctree)/scripts/generate_rust_analyzer.py \
> > +               $(abs_srctree) $(abs_objtree) \
> > +               $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \
> > +               $(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json
> 
> 
> 
> This is equivalent to:
> 
>   >  $(extmod_prefix)/rust-project.json
> 
> 
> 
> See the rule of 'compile_commands.json'.
> 
> 
> 
> 
> 
> 
> 
> >  $(obj)/core.o: private skip_clippy = 1
> >  $(obj)/core.o: private skip_flags = -Dunreachable_pub
> > diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
> > index ecc7ea9a4dcf..1792f379ee4e 100755
> > --- a/scripts/generate_rust_analyzer.py
> > +++ b/scripts/generate_rust_analyzer.py
> > @@ -6,10 +6,11 @@
> >  import argparse
> >  import json
> >  import logging
> > +import os
> >  import pathlib
> >  import sys
> >
> > -def generate_crates(srctree, objtree, sysroot_src):
> > +def generate_crates(srctree, objtree, sysroot_src, external_src):
> >      # Generate the configuration list.
> >      cfg = []
> >      with open(objtree / "include" / "generated" / "rustc_cfg") as fd:
> > @@ -65,7 +66,7 @@ def generate_crates(srctree, objtree, sysroot_src):
> >          [],
> >          is_proc_macro=True,
> >      )
> > -    crates[-1]["proc_macro_dylib_path"] = "rust/libmacros.so"
> > +    crates[-1]["proc_macro_dylib_path"] = f"{objtree}/rust/libmacros.so"
> >
> >      append_crate(
> >          "build_error",
> > @@ -95,25 +96,28 @@ def generate_crates(srctree, objtree, sysroot_src):
> >          "exclude_dirs": [],
> >      }
> >
> > +    def is_root_crate(build_file, target):
> > +        return os.path.exists(build_file) and target in open(build_file).read()
> > +
> >      # Then, the rest outside of `rust/`.
> >      #
> >      # We explicitly mention the top-level folders we want to cover.
> 
> 
> Huh, not maintainable, unfortunately.
> 
> 
> 
> 
> 
> > -    for folder in ("samples", "drivers"):
> > +    for folder in ("samples", "drivers") if external_src is None else [external_src]:
> >          for path in (srctree / folder).rglob("*.rs"):
> 
> 
> 
> It is odd to add 'srctree' prefix to external module sources.
For external module sources, external_src is an absolute path and hence
srctree is ignored in this call.
> 
> 
> 
> I think rust-project.json is a similar concept to
> compile_commands.json, but it was implemented
> in a different way.
> 
I have not included the changes mentioned to refactor the rust-analyzer
target along the lines of how compile_commands.json has been solved
since it was beyond the scope of this changeset. However, I can take
this up as a follow up changeset.
> 
> 
> 
> 
> 
> 
> -- 
> Best Regards
> Masahiro Yamada