* [PATCH v1 0/1] scripts: Fix "make gtags" for O= kernel builds @ 2023-05-04 20:18 Ahmed S. Darwish 2023-05-04 20:18 ` [PATCH v1 1/1] scripts/tags.sh: Fix gtags generation " Ahmed S. Darwish 2023-05-09 1:26 ` [PATCH v2 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish 0 siblings, 2 replies; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-04 20:18 UTC (permalink / raw) To: Masahiro Yamada, Nathan Chancellor, Nick Desaulniers, Nicolas Schier Cc: Thomas Gleixner, linux-kbuild, LKML, Ahmed S. Darwish Hi, make gtags for O= kernel builds is currently broken. For example, when doing: make O=../build/ x86_64_defconfig make O=../build/ gtags gtags generates a warning for each kernel source file to be indexed: make[1]: Entering directory '/home/darwi/build' GEN gtags Warning: '/home/darwi/linux/arch/x86/include/asm/qspinlock.h' is out of source tree. ignored. Warning: '/home/darwi/linux/arch/x86/include/asm/hpet.h' is out of source tree. ignored. ... Warning: '/home/darwi/linux/virt/lib/irqbypass.c' is out of source tree. ignored. make[1]: Leaving directory '/home/darwi/build/' and then generates an empty index: $ du -hs ~/build/G* 16K /home/darwi/build/GPATH 16K /home/darwi/build/GRTAGS 16K /home/darwi/build/GTAGS This series includes a proposed fix. After applying it: $ make O=../build/ gtags make[1]: Entering directory '/home/darwi/build' GEN gtags make[1]: Leaving directory '/home/darwi/build' $ du -hs ~/build/G* 9.1M /home/darwi/build/GPATH 506M /home/darwi/build/GRTAGS 696M /home/darwi/build/GTAGS The generated files can then be integrated with editors or IDEs as usual. Thanks, => Ahmed S. Darwish (1): scripts/tags.sh: Fix gtags generation for O= kernel builds scripts/tags.sh | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) base-commit: 1a5304fecee523060f26e2778d9d8e33c0562df3 -- 2.30.2 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v1 1/1] scripts/tags.sh: Fix gtags generation for O= kernel builds 2023-05-04 20:18 [PATCH v1 0/1] scripts: Fix "make gtags" for O= kernel builds Ahmed S. Darwish @ 2023-05-04 20:18 ` Ahmed S. Darwish 2023-05-04 21:32 ` Nathan Chancellor 2023-05-09 1:26 ` [PATCH v2 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish 1 sibling, 1 reply; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-04 20:18 UTC (permalink / raw) To: Masahiro Yamada, Nathan Chancellor, Nick Desaulniers, Nicolas Schier Cc: Thomas Gleixner, linux-kbuild, LKML, Ahmed S. Darwish gtags considers any file outside of its current working directory "outside the source tree" and refuses to index it. For O= kernel builds, scripts/tags.sh invokes gtags with the current working directory set to ${O}. This leads to gtags ignoring the entire kernel source and generating an empty index. For O= builds, set gtags' working directory to the kernel source tree and explicitly set its output path through parameters instead. Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de> --- scripts/tags.sh | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/scripts/tags.sh b/scripts/tags.sh index ea31640b2671..1a6db535503b 100755 --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -131,7 +131,14 @@ docscope() dogtags() { - all_target_sources | gtags -i -f - + # gtags refuses to index any file outside of the current working + # directory. For O= builds, set the current working directory to + # the kernel source tree and the output tags dir to ${O}. + suffixparams= + if [ -v O ]; then + suffixparams="-C $tree $O" + fi + all_target_sources | gtags -i -f - $suffixparams } # Basic regular expressions with an optional /kind-spec/ for ctags and -- 2.30.2 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH v1 1/1] scripts/tags.sh: Fix gtags generation for O= kernel builds 2023-05-04 20:18 ` [PATCH v1 1/1] scripts/tags.sh: Fix gtags generation " Ahmed S. Darwish @ 2023-05-04 21:32 ` Nathan Chancellor 2023-05-04 22:00 ` Ahmed S. Darwish 0 siblings, 1 reply; 17+ messages in thread From: Nathan Chancellor @ 2023-05-04 21:32 UTC (permalink / raw) To: Ahmed S. Darwish Cc: Masahiro Yamada, Nick Desaulniers, Nicolas Schier, Thomas Gleixner, linux-kbuild, LKML On Thu, May 04, 2023 at 10:18:33PM +0200, Ahmed S. Darwish wrote: > gtags considers any file outside of its current working directory > "outside the source tree" and refuses to index it. > > For O= kernel builds, scripts/tags.sh invokes gtags with the current > working directory set to ${O}. This leads to gtags ignoring the entire > kernel source and generating an empty index. > > For O= builds, set gtags' working directory to the kernel source tree > and explicitly set its output path through parameters instead. > > Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de> Reviewed-by: Nathan Chancellor <nathan@kernel.org> > --- > scripts/tags.sh | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/scripts/tags.sh b/scripts/tags.sh > index ea31640b2671..1a6db535503b 100755 > --- a/scripts/tags.sh > +++ b/scripts/tags.sh > @@ -131,7 +131,14 @@ docscope() > > dogtags() > { > - all_target_sources | gtags -i -f - > + # gtags refuses to index any file outside of the current working > + # directory. For O= builds, set the current working directory to > + # the kernel source tree and the output tags dir to ${O}. > + suffixparams= > + if [ -v O ]; then I think if [ -n "$O" ]; then would match the style preferred by Kbuild (though that is usually for portability sake, which probably does not matter here since bash is explicitly requested). Perhaps not worth addressing if there is no other reason for a v2. > + suffixparams="-C $tree $O" > + fi > + all_target_sources | gtags -i -f - $suffixparams > } > > # Basic regular expressions with an optional /kind-spec/ for ctags and > -- > 2.30.2 > ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v1 1/1] scripts/tags.sh: Fix gtags generation for O= kernel builds 2023-05-04 21:32 ` Nathan Chancellor @ 2023-05-04 22:00 ` Ahmed S. Darwish 2023-05-05 5:13 ` Masahiro Yamada 0 siblings, 1 reply; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-04 22:00 UTC (permalink / raw) To: Nathan Chancellor Cc: Masahiro Yamada, Nick Desaulniers, Nicolas Schier, Thomas Gleixner, linux-kbuild, LKML Hi Nathan, On Thu, 04 May 2023, Nathan Chancellor wrote: > > On Thu, May 04, 2023 at 10:18:33PM +0200, Ahmed S. Darwish wrote: ... > > + suffixparams= > > + if [ -v O ]; then > > I think > > if [ -n "$O" ]; then > > would match the style preferred by Kbuild (though that is usually for > portability sake, which probably does not matter here since bash is > explicitly requested). Perhaps not worth addressing if there is no other > reason for a v2. > Thanks, I'll do it. I've just discovered that a v2 is necessary anyway. If O= has a "~", for example as in: make O=~/build/ gtags the snippet below: > > + suffixparams="-C $tree $O" > > + fi > > + all_target_sources | gtags -i -f - $suffixparams ^ will fail since the "~" in the O= directory path won't get dereferenced before getting passed to the gtags call (an eval is needed). I'll submit a v2 shortly. Kind regards, -- Ahmed S. Darwish Linutronix GmbH ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v1 1/1] scripts/tags.sh: Fix gtags generation for O= kernel builds 2023-05-04 22:00 ` Ahmed S. Darwish @ 2023-05-05 5:13 ` Masahiro Yamada 2023-05-05 5:17 ` Masahiro Yamada 2023-05-08 14:11 ` Ahmed S. Darwish 0 siblings, 2 replies; 17+ messages in thread From: Masahiro Yamada @ 2023-05-05 5:13 UTC (permalink / raw) To: Ahmed S. Darwish Cc: Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Thomas Gleixner, linux-kbuild, LKML On Fri, May 5, 2023 at 12:00 AM Ahmed S. Darwish <darwi@linutronix.de> wrote: > > Hi Nathan, > > On Thu, 04 May 2023, Nathan Chancellor wrote: > > > > On Thu, May 04, 2023 at 10:18:33PM +0200, Ahmed S. Darwish wrote: > ... > > > + suffixparams= > > > + if [ -v O ]; then > > > > I think > > > > if [ -n "$O" ]; then > > > > would match the style preferred by Kbuild (though that is usually for > > portability sake, which probably does not matter here since bash is > > explicitly requested). Perhaps not worth addressing if there is no other > > reason for a v2. > > > > Thanks, I'll do it. I've just discovered that a v2 is necessary anyway. > > If O= has a "~", for example as in: > > make O=~/build/ gtags > > the snippet below: > > > > + suffixparams="-C $tree $O" > > > + fi > > > + all_target_sources | gtags -i -f - $suffixparams > ^ > will fail since the "~" in the O= directory path won't get dereferenced > before getting passed to the gtags call (an eval is needed). > > I'll submit a v2 shortly. > > Kind regards, > > -- > Ahmed S. Darwish > Linutronix GmbH It is wrong to check whether you are building out of the source tree. See line 159 of the Makefile. BTW, this patch does not work for me. It spits a ton of "not found" warnings, then generates empty tags. $ make O=build gtags make[1]: Entering directory '/home/masahiro/ref/linux/build' GEN gtags Warning: '../arch/x86/include/asm/vmalloc.h' not found. ignored. Warning: '../arch/x86/include/asm/pgtable-3level_types.h' not found. ignored. Warning: '../arch/x86/include/asm/paravirt.h' not found. ignored. Warning: '../arch/x86/include/asm/text-patching.h' not found. ignored. Warning: '../arch/x86/include/asm/softirq_stack.h' not found. ignored. Warning: '../arch/x86/include/asm/intel_ds.h' not found. ignored. Warning: '../arch/x86/include/asm/resctrl.h' not found. ignored. Warning: '../arch/x86/include/asm/setup_arch.h' not found. ignored. Warning: '../arch/x86/include/asm/simd.h' not found. ignored. Warning: '../arch/x86/include/asm/mmconfig.h' not found. ignored. Warning: '../arch/x86/include/asm/pgtable_types.h' not found. ignored. Warning: '../arch/x86/include/asm/mem_encrypt.h' not found. ignored. Warning: '../arch/x86/include/asm/dmi.h' not found. ignored. Warning: '../arch/x86/include/asm/thermal.h' not found. ignored. ... -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v1 1/1] scripts/tags.sh: Fix gtags generation for O= kernel builds 2023-05-05 5:13 ` Masahiro Yamada @ 2023-05-05 5:17 ` Masahiro Yamada 2023-05-08 14:11 ` Ahmed S. Darwish 1 sibling, 0 replies; 17+ messages in thread From: Masahiro Yamada @ 2023-05-05 5:17 UTC (permalink / raw) To: Ahmed S. Darwish Cc: Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Thomas Gleixner, linux-kbuild, LKML On Fri, May 5, 2023 at 7:13 AM Masahiro Yamada <masahiroy@kernel.org> wrote: > > On Fri, May 5, 2023 at 12:00 AM Ahmed S. Darwish <darwi@linutronix.de> wrote: > > > > Hi Nathan, > > > > On Thu, 04 May 2023, Nathan Chancellor wrote: > > > > > > On Thu, May 04, 2023 at 10:18:33PM +0200, Ahmed S. Darwish wrote: > > ... > > > > + suffixparams= > > > > + if [ -v O ]; then > > > > > > I think > > > > > > if [ -n "$O" ]; then > > > > > > would match the style preferred by Kbuild (though that is usually for > > > portability sake, which probably does not matter here since bash is > > > explicitly requested). Perhaps not worth addressing if there is no other > > > reason for a v2. > > > > > > > Thanks, I'll do it. I've just discovered that a v2 is necessary anyway. > > > > If O= has a "~", for example as in: > > > > make O=~/build/ gtags > > > > the snippet below: > > > > > > + suffixparams="-C $tree $O" > > > > + fi > > > > + all_target_sources | gtags -i -f - $suffixparams > > ^ > > will fail since the "~" in the O= directory path won't get dereferenced > > before getting passed to the gtags call (an eval is needed). > > > > I'll submit a v2 shortly. > > > > Kind regards, > > > > -- > > Ahmed S. Darwish > > Linutronix GmbH > > > > It is wrong to check whether you are building out of the > source tree. See line 159 of the Makefile. Let me correct this sentense. It is wrong to use 'O' to check whether you are building out of the source tree. -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v1 1/1] scripts/tags.sh: Fix gtags generation for O= kernel builds 2023-05-05 5:13 ` Masahiro Yamada 2023-05-05 5:17 ` Masahiro Yamada @ 2023-05-08 14:11 ` Ahmed S. Darwish 1 sibling, 0 replies; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-08 14:11 UTC (permalink / raw) To: Masahiro Yamada Cc: Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Thomas Gleixner, linux-kbuild, LKML Hi Masahiro, On Fri, 05 May 2023, Masahiro Yamada wrote: > > It is wrong to check whether you are building out of the > source tree. See line 159 of the Makefile. > Oh, didn't think about that case. Thanks for the reference and the further clarification in reply. I'll remove the ${O} check then and use saner mechanisms. > BTW, this patch does not work for me. > It spits a ton of "not found" warnings, then generates > empty tags. > > > $ make O=build gtags Interesting... When doing: $ make O=../build gtags scripts/tags.sh "$tree" variable is set to the absolute path of the kernel source tree. Thus all the paths fed to gtags are absolute and this patch series works. When doing what you tested with: $ make O=build/ gtags scripts/tags.sh "$tree" variable is set to the path of the kernel source tree *relative* to O=build/. So in that case kernel source "$tree" equals ".." With this series, the build will fail as gtags current working dir is the kernel source tree, and all the fed paths are thus invalid as they're relative to O=build/ instead. Without this series the build will still fail given the original problem of having the files "outside the source tree", where gtags thinks the source tree is "build/". I'll think of something that can cover the both cases. Kind regards, Ahmed -- Linutronix GmbH ^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v2 0/2] scripts: Resolve gtags empty index generation 2023-05-04 20:18 [PATCH v1 0/1] scripts: Fix "make gtags" for O= kernel builds Ahmed S. Darwish 2023-05-04 20:18 ` [PATCH v1 1/1] scripts/tags.sh: Fix gtags generation " Ahmed S. Darwish @ 2023-05-09 1:26 ` Ahmed S. Darwish 2023-05-09 1:26 ` [PATCH v2 1/2] scripts/tags.sh: " Ahmed S. Darwish ` (2 more replies) 1 sibling, 3 replies; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-09 1:26 UTC (permalink / raw) To: Masahiro Yamada, Nathan Chancellor, Nick Desaulniers, Nicolas Schier Cc: Thomas Gleixner, linux-kbuild, LKML, Ahmed S. Darwish Hi, v2-changelog ------------ Handle review remarks from Masahiro Yamada: - scripts/tags.sh: remove the O= language, and focus on the general case of the build directory being different from the kernel source tree, as specified in kernel Makefile L159. - Fix failure when build directory is a subdirectory of the kernel source tree. NEW: - Update Documentation/process/changes.rst with new gtags (GNU GLOBAL) requirements. Thanks! Cover letter / v1 ----------------- https://lkml.kernel.org/r/20230504201833.202494-1-darwi@linutronix.de make gtags for O= kernel builds is currently broken. For example, when doing: make O=../build/ x86_64_defconfig make O=../build/ gtags gtags generates a warning for each kernel source file to be indexed: make[1]: Entering directory '/home/darwi/build' GEN gtags Warning: '/home/darwi/linux/arch/x86/include/asm/qspinlock.h' is out of source tree. ignored. Warning: '/home/darwi/linux/arch/x86/include/asm/hpet.h' is out of source tree. ignored. ... Warning: '/home/darwi/linux/virt/lib/irqbypass.c' is out of source tree. ignored. make[1]: Leaving directory '/home/darwi/build/' and then generates an empty index: $ du -hs ~/build/G* 16K /home/darwi/build/GPATH 16K /home/darwi/build/GRTAGS 16K /home/darwi/build/GTAGS This series includes a proposed fix. After applying it: $ make O=../build/ gtags make[1]: Entering directory '/home/darwi/build' GEN gtags make[1]: Leaving directory '/home/darwi/build' $ du -hs ~/build/G* 9.1M /home/darwi/build/GPATH 506M /home/darwi/build/GRTAGS 696M /home/darwi/build/GTAGS The generated files can then be integrated with editors or IDEs as usual. => Ahmed S. Darwish (2): scripts/tags.sh: Resolve gtags empty index generation docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 Documentation/process/changes.rst | 7 +++++++ scripts/tags.sh | 14 +++++++++++++- 2 files changed, 20 insertions(+), 1 deletion(-) base-commit: ba0ad6ed89fd5dada3b7b65ef2b08e95d449d4ab -- 2.40.0 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v2 1/2] scripts/tags.sh: Resolve gtags empty index generation 2023-05-09 1:26 ` [PATCH v2 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish @ 2023-05-09 1:26 ` Ahmed S. Darwish 2023-05-11 18:51 ` Masahiro Yamada 2023-05-09 1:26 ` [PATCH v2 2/2] docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 Ahmed S. Darwish 2023-05-15 17:32 ` [PATCH v3 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish 2 siblings, 1 reply; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-09 1:26 UTC (permalink / raw) To: Masahiro Yamada, Nathan Chancellor, Nick Desaulniers, Nicolas Schier Cc: Thomas Gleixner, linux-kbuild, LKML, Ahmed S. Darwish gtags considers any file outside of its current working directory "outside the source tree" and refuses to index it. For O= kernel builds, or when "make" is invoked from a directory other then the kernel source tree, gtags ignores the entire kernel source and generates an empty index. Force-set gtags current working directory to the kernel source tree. Due to commit 9da0763bdd82 ("kbuild: Use relative path when building in a subdir of the source tree"), if the kernel build is done in a sub-directory of the kernel source tree, the kernel Makefile will set the kernel's $srctree to ".." for shorter compile-time and run-time warnings. Consequently, the list of files to be indexed will be in the "../*" form, rendering all such paths invalid once gtags switches to the kernel source tree as its current working directory. If gtags indexing is requested and the build directory is not the kernel source tree, index all files in absolute-path form. Note, indexing in absolute-path form will not affect the generated index, as paths in gtags indices are always relative to the gtags "root directory" (as evidenced by "gtags --dump"). Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de> Cc: <stable@vger.kernel.org> --- scripts/tags.sh | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/scripts/tags.sh b/scripts/tags.sh index ea31640b2671..3de4b4ebd891 100755 --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -32,6 +32,14 @@ else tree=${srctree}/ fi + +# gtags(1) refuses to index any file outside of its current working dir. +# If gtags indexing is requested and the build output directory is not +# the kernel source tree, index all files in absolute-path form. +if [ "$1" = "gtags" -a -n "${tree}" ]; then + tree=$(realpath $tree)/ +fi + # Detect if ALLSOURCE_ARCHS is set. If not, we assume SRCARCH if [ "${ALLSOURCE_ARCHS}" = "" ]; then ALLSOURCE_ARCHS=${SRCARCH} @@ -131,7 +139,11 @@ docscope() dogtags() { - all_target_sources | gtags -i -f - + local gtagsoutdir="${PWD}" + local gtagsroot="${tree}" + + [ -z "${gtagsroot}" ] && gtagsroot="." + all_target_sources | gtags -i -C $gtagsroot -f - $gtagsoutdir } # Basic regular expressions with an optional /kind-spec/ for ctags and -- 2.40.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH v2 1/2] scripts/tags.sh: Resolve gtags empty index generation 2023-05-09 1:26 ` [PATCH v2 1/2] scripts/tags.sh: " Ahmed S. Darwish @ 2023-05-11 18:51 ` Masahiro Yamada 2023-05-15 15:23 ` Ahmed S. Darwish 0 siblings, 1 reply; 17+ messages in thread From: Masahiro Yamada @ 2023-05-11 18:51 UTC (permalink / raw) To: Ahmed S. Darwish Cc: Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Thomas Gleixner, linux-kbuild, LKML On Tue, May 9, 2023 at 10:26 AM Ahmed S. Darwish <darwi@linutronix.de> wrote: > > gtags considers any file outside of its current working directory > "outside the source tree" and refuses to index it. For O= kernel builds, > or when "make" is invoked from a directory other then the kernel source > tree, gtags ignores the entire kernel source and generates an empty > index. > > Force-set gtags current working directory to the kernel source tree. > > Due to commit 9da0763bdd82 ("kbuild: Use relative path when building in > a subdir of the source tree"), if the kernel build is done in a > sub-directory of the kernel source tree, the kernel Makefile will set > the kernel's $srctree to ".." for shorter compile-time and run-time > warnings. Consequently, the list of files to be indexed will be in the > "../*" form, rendering all such paths invalid once gtags switches to the > kernel source tree as its current working directory. > > If gtags indexing is requested and the build directory is not the kernel > source tree, index all files in absolute-path form. > > Note, indexing in absolute-path form will not affect the generated > index, as paths in gtags indices are always relative to the gtags "root > directory" (as evidenced by "gtags --dump"). The code works as claimed, but I am just curious. If all the paths are relative, how can you use the tags files located in a separate directory? "make O=foo gtags" creates tags files in foo/. I want to use them from emacs. emacs cannot find the right file because it assumes the path is relative to 'foo' instead of the source tree. I set GTAGSROOT to the source tree, but I could not find a way to use it in a useful way. > diff --git a/scripts/tags.sh b/scripts/tags.sh > index ea31640b2671..3de4b4ebd891 100755 > --- a/scripts/tags.sh > +++ b/scripts/tags.sh > @@ -32,6 +32,14 @@ else > tree=${srctree}/ > fi > > + Unneeded empty line addition. > +# gtags(1) refuses to index any file outside of its current working dir. > +# If gtags indexing is requested and the build output directory is not > +# the kernel source tree, index all files in absolute-path form. > +if [ "$1" = "gtags" -a -n "${tree}" ]; then > + tree=$(realpath $tree)/ I decided to run shellcheck for new code. Please follow the suggestion from the tool. In scripts/tags.sh line 40: tree=$(realpath $tree)/ ^---^ SC2086 (info): Double quote to prevent globbing and word splitting. Did you mean: tree=$(realpath "$tree")/ (You do not need to fix the entire script. This is only for new code). > @@ -131,7 +139,11 @@ docscope() > > dogtags() > { > - all_target_sources | gtags -i -f - > + local gtagsoutdir="${PWD}" > + local gtagsroot="${tree}" > + > + [ -z "${gtagsroot}" ] && gtagsroot="." > + all_target_sources | gtags -i -C $gtagsroot -f - $gtagsoutdir > } You can write it in one line. dogtags() { all_target_sources | gtags -i -C "${tree:-.}" -f - "${PWD}" } -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 1/2] scripts/tags.sh: Resolve gtags empty index generation 2023-05-11 18:51 ` Masahiro Yamada @ 2023-05-15 15:23 ` Ahmed S. Darwish 2023-05-15 16:35 ` Ahmed S. Darwish 0 siblings, 1 reply; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-15 15:23 UTC (permalink / raw) To: Masahiro Yamada Cc: Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Thomas Gleixner, linux-kbuild, LKML On Fri, 12 May 2023, Masahiro Yamada wrote: > > The code works as claimed, but I am just curious. Thanks. > If all the paths are relative, how can you use the tags files located > in a separate directory? > > "make O=foo gtags" creates tags files in foo/. > I want to use them from emacs. > emacs cannot find the right file because > it assumes the path is relative to 'foo' instead of the source tree. > Correct. In theory, since all the indexed linux source tree paths at the gtags generated files (GPATH/GRTAGS/GTAGS) are relative to the linux source tree root, opening such files from emacs ggtags-mode, *wherever* these files are, "should" work. In practice, ggtags/global (and thus also emacs ggtags-mode) operate with a model of the world where all indexed files must be under the source "root directory". So, the GPATH/GRTAGS/GTAGS files are expected to be under the source tree (except in special GTAGSLIBPATH= cases). > I set GTAGSROOT to the source tree, but I could not find a way > to use it in a useful way. Yes, that won't work, as emacs will search for the G* database files under that folder instead. Meanwhile setting GTAGSROOT to the O= directory, or to a build directory that is different from the kernel source tree, as in: cd ~/linux O=~/build/build-linux-x86 make O=$O x86_64_defconfig make O=$O gtags GTAGSROOT=$O emacs init/main.c will "mostly" succeed (as you hinted at): M-x ggtags-mode # emacs finds gtags files under ${GTAGSROOT} and sets ${GTAGSROOT} as # the root of the project M-x ggtags-find-definition Definition: rcu_read_lock -*- mode: ggtags-global; default-directory: "~/build/build-linux-x86/" -*- Global started at Mon May 15 15:54:01 global -v --result=grep --color=always --path-style=shorter -- rcu_read_lock include/linux/rcupdate.h:769:static __always_inline void rcu_read_lock(void) 1 object located (using '/home/darwi/build/build-linux-x86/GTAGS'). Global found 1 definition at Mon May 15 15:54:01 # Prompt Find this match in (default include/linux/rcupdate.h)?: ~/build/build-linux-x86/ ^^^ But at the Prompt step above, things break. In a fully working setup, this prompt will not be shown and emacs just jumps to rcuupdate.h line 769. What I personally do to mitigate that problem is: cd ~/linux for f in GTAGS GRTAGS GPATH; do ln -vsf ${O}/$f . done and switch these symlinks through minor local shell plumbing whenever I'm switching kernel projects with different build directories. It is not ideal, but maybe we can discuss this with the global(1) people at a later step. At least with this patch series, "make O=xyz/ gtags" produces a valid index. > > > +# gtags(1) refuses to index any file outside of its current working dir. > > +# If gtags indexing is requested and the build output directory is not > > +# the kernel source tree, index all files in absolute-path form. > > +if [ "$1" = "gtags" -a -n "${tree}" ]; then > > + tree=$(realpath $tree)/ > > I decided to run shellcheck for new code. > Please follow the suggestion from the tool. > > In scripts/tags.sh line 40: > tree=$(realpath $tree)/ > ^---^ SC2086 (info): Double quote to prevent > globbing and word splitting. > > Did you mean: > tree=$(realpath "$tree")/ > > (You do not need to fix the entire script. > This is only for new code). > Yes, the reason was to following the existing coding pattern at scripts/tags.sh. But, sure, will do. > > > @@ -131,7 +139,11 @@ docscope() > > > > dogtags() > > { > > - all_target_sources | gtags -i -f - > > + local gtagsoutdir="${PWD}" > > + local gtagsroot="${tree}" > > + > > + [ -z "${gtagsroot}" ] && gtagsroot="." > > + all_target_sources | gtags -i -C $gtagsroot -f - $gtagsoutdir > > } > > You can write it in one line. > > dogtags() > { > all_target_sources | gtags -i -C "${tree:-.}" -f - "${PWD}" > } > Ditto. The script was almost-fully POSIX style (except the first line), so I avoided bash features on purpose. I personlly always prefer using Bash features though, so I'll definitely update the code. Thanks a lot for the review. I'll send a v3. -- Ahmed S. Darwish Linutronix GmbH ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 1/2] scripts/tags.sh: Resolve gtags empty index generation 2023-05-15 15:23 ` Ahmed S. Darwish @ 2023-05-15 16:35 ` Ahmed S. Darwish 0 siblings, 0 replies; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-15 16:35 UTC (permalink / raw) To: Masahiro Yamada Cc: Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Thomas Gleixner, linux-kbuild, LKML On Mon, 15 May 2023, Ahmed S. Darwish wrote: > On Fri, 12 May 2023, Masahiro Yamada wrote: > > > > You can write it in one line. > > > > dogtags() > > { > > all_target_sources | gtags -i -C "${tree:-.}" -f - "${PWD}" > > } > > > > Ditto. The script was almost-fully POSIX style (except the first line), > so I avoided bash features on purpose. > Nitpick for correctness sake the "Use default values" parameter expansion is actually POSIX-ly correct: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_02 Thanks, Ahmed ^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v2 2/2] docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 2023-05-09 1:26 ` [PATCH v2 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish 2023-05-09 1:26 ` [PATCH v2 1/2] scripts/tags.sh: " Ahmed S. Darwish @ 2023-05-09 1:26 ` Ahmed S. Darwish 2023-05-15 17:32 ` [PATCH v3 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish 2 siblings, 0 replies; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-09 1:26 UTC (permalink / raw) To: Masahiro Yamada, Nathan Chancellor, Nick Desaulniers, Nicolas Schier Cc: Thomas Gleixner, linux-kbuild, LKML, Ahmed S. Darwish Kernel build now uses the gtags "-C (--directory)" option, available since GNU GLOBAL v6.6.5. Update the documentation accordingly. Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de> Cc: <stable@vger.kernel.org> Link: https://lists.gnu.org/archive/html/info-global/2020-09/msg00000.html --- Documentation/process/changes.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index ef540865ad22..a9ef00509c9b 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -60,6 +60,7 @@ openssl & libcrypto 1.0.0 openssl version bc 1.06.95 bc --version Sphinx\ [#f1]_ 1.7 sphinx-build --version cpio any cpio --version +gtags (optional) 6.6.5 gtags --version ====================== =============== ======================================== .. [#f1] Sphinx is needed only to build the Kernel documentation @@ -174,6 +175,12 @@ You will need openssl to build kernels 3.7 and higher if module signing is enabled. You will also need openssl development packages to build kernels 4.3 and higher. +gtags / GNU GLOBAL (optional) +----------------------------- + +The kernel build requires GNU GLOBAL version 6.6.5 or later to generate +tag files through ``make gtags``. This is due to its use of the gtags +``-C (--directory)`` flag. System utilities **************** -- 2.40.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 0/2] scripts: Resolve gtags empty index generation 2023-05-09 1:26 ` [PATCH v2 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish 2023-05-09 1:26 ` [PATCH v2 1/2] scripts/tags.sh: " Ahmed S. Darwish 2023-05-09 1:26 ` [PATCH v2 2/2] docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 Ahmed S. Darwish @ 2023-05-15 17:32 ` Ahmed S. Darwish 2023-05-15 17:32 ` [PATCH v3 1/2] scripts/tags.sh: " Ahmed S. Darwish ` (2 more replies) 2 siblings, 3 replies; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-15 17:32 UTC (permalink / raw) To: Masahiro Yamada, Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Jonathan Corbet Cc: Thomas Gleixner, linux-kbuild, linux-doc, LKML, Ahmed S. Darwish Hi, v3-changelog ------------ Handle review remarks from Masahiro Yamada: - Apply shellcheck on new "scripts/tags.sh" code. - Shorten code through shell's "default value" parameter expansion. NEW: - Cc docs maintainer (Documentation/process/changes.rst change). Thanks! v2-changelog ------------ https://lkml.kernel.org/r/20230509012616.81579-1-darwi@linutronix.de Handle review remarks from Masahiro Yamada: - scripts/tags.sh: remove the O= language, and focus on the general case of the build directory being different from the kernel source tree, as specified in kernel Makefile L159. - Fix failure when build directory is a subdirectory of the kernel source tree. NEW: - Update Documentation/process/changes.rst with new gtags (GNU GLOBAL) requirements. Thanks! Cover letter / v1 ----------------- https://lkml.kernel.org/r/20230504201833.202494-1-darwi@linutronix.de make gtags for O= kernel builds is currently broken. For example, when doing: make O=../build/ x86_64_defconfig make O=../build/ gtags gtags generates a warning for each kernel source file to be indexed: make[1]: Entering directory '/home/darwi/build' GEN gtags Warning: '/home/darwi/linux/arch/x86/include/asm/qspinlock.h' is out of source tree. ignored. Warning: '/home/darwi/linux/arch/x86/include/asm/hpet.h' is out of source tree. ignored. ... Warning: '/home/darwi/linux/virt/lib/irqbypass.c' is out of source tree. ignored. make[1]: Leaving directory '/home/darwi/build/' and then generates an empty index: $ du -hs ~/build/G* 16K /home/darwi/build/GPATH 16K /home/darwi/build/GRTAGS 16K /home/darwi/build/GTAGS This series includes a proposed fix. After applying it: $ make O=../build/ gtags make[1]: Entering directory '/home/darwi/build' GEN gtags make[1]: Leaving directory '/home/darwi/build' $ du -hs ~/build/G* 9.1M /home/darwi/build/GPATH 506M /home/darwi/build/GRTAGS 696M /home/darwi/build/GTAGS The generated files can then be integrated with editors or IDEs as usual. => Ahmed S. Darwish (2): scripts/tags.sh: Resolve gtags empty index generation docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 Documentation/process/changes.rst | 7 +++++++ scripts/tags.sh | 9 ++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) base-commit: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6 -- 2.30.2 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v3 1/2] scripts/tags.sh: Resolve gtags empty index generation 2023-05-15 17:32 ` [PATCH v3 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish @ 2023-05-15 17:32 ` Ahmed S. Darwish 2023-05-15 17:32 ` [PATCH v3 2/2] docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 Ahmed S. Darwish 2023-05-20 22:41 ` [PATCH v3 0/2] scripts: Resolve gtags empty index generation Masahiro Yamada 2 siblings, 0 replies; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-15 17:32 UTC (permalink / raw) To: Masahiro Yamada, Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Jonathan Corbet Cc: Thomas Gleixner, linux-kbuild, linux-doc, LKML, Ahmed S. Darwish gtags considers any file outside of its current working directory "outside the source tree" and refuses to index it. For O= kernel builds, or when "make" is invoked from a directory other then the kernel source tree, gtags ignores the entire kernel source and generates an empty index. Force-set gtags current working directory to the kernel source tree. Due to commit 9da0763bdd82 ("kbuild: Use relative path when building in a subdir of the source tree"), if the kernel build is done in a sub-directory of the kernel source tree, the kernel Makefile will set the kernel's $srctree to ".." for shorter compile-time and run-time warnings. Consequently, the list of files to be indexed will be in the "../*" form, rendering all such paths invalid once gtags switches to the kernel source tree as its current working directory. If gtags indexing is requested and the build directory is not the kernel source tree, index all files in absolute-path form. Note, indexing in absolute-path form will not affect the generated index, as paths in gtags indices are always relative to the gtags "root directory" anyway (as evidenced by "gtags --dump"). Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de> Cc: <stable@vger.kernel.org> --- scripts/tags.sh | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/scripts/tags.sh b/scripts/tags.sh index ea31640b2671..f6b3c7cd39c7 100755 --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -32,6 +32,13 @@ else tree=${srctree}/ fi +# gtags(1) refuses to index any file outside of its current working dir. +# If gtags indexing is requested and the build output directory is not +# the kernel source tree, index all files in absolute-path form. +if [[ "$1" == "gtags" && -n "${tree}" ]]; then + tree=$(realpath "$tree")/ +fi + # Detect if ALLSOURCE_ARCHS is set. If not, we assume SRCARCH if [ "${ALLSOURCE_ARCHS}" = "" ]; then ALLSOURCE_ARCHS=${SRCARCH} @@ -131,7 +138,7 @@ docscope() dogtags() { - all_target_sources | gtags -i -f - + all_target_sources | gtags -i -C "${tree:-.}" -f - "$PWD" } # Basic regular expressions with an optional /kind-spec/ for ctags and -- 2.30.2 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 2/2] docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 2023-05-15 17:32 ` [PATCH v3 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish 2023-05-15 17:32 ` [PATCH v3 1/2] scripts/tags.sh: " Ahmed S. Darwish @ 2023-05-15 17:32 ` Ahmed S. Darwish 2023-05-20 22:41 ` [PATCH v3 0/2] scripts: Resolve gtags empty index generation Masahiro Yamada 2 siblings, 0 replies; 17+ messages in thread From: Ahmed S. Darwish @ 2023-05-15 17:32 UTC (permalink / raw) To: Masahiro Yamada, Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Jonathan Corbet Cc: Thomas Gleixner, linux-kbuild, linux-doc, LKML, Ahmed S. Darwish Kernel build now uses the gtags "-C (--directory)" option, available since GNU GLOBAL v6.6.5. Update the documentation accordingly. Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de> Cc: <stable@vger.kernel.org> Link: https://lists.gnu.org/archive/html/info-global/2020-09/msg00000.html --- Documentation/process/changes.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index ef540865ad22..a9ef00509c9b 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -60,6 +60,7 @@ openssl & libcrypto 1.0.0 openssl version bc 1.06.95 bc --version Sphinx\ [#f1]_ 1.7 sphinx-build --version cpio any cpio --version +gtags (optional) 6.6.5 gtags --version ====================== =============== ======================================== .. [#f1] Sphinx is needed only to build the Kernel documentation @@ -174,6 +175,12 @@ You will need openssl to build kernels 3.7 and higher if module signing is enabled. You will also need openssl development packages to build kernels 4.3 and higher. +gtags / GNU GLOBAL (optional) +----------------------------- + +The kernel build requires GNU GLOBAL version 6.6.5 or later to generate +tag files through ``make gtags``. This is due to its use of the gtags +``-C (--directory)`` flag. System utilities **************** -- 2.30.2 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH v3 0/2] scripts: Resolve gtags empty index generation 2023-05-15 17:32 ` [PATCH v3 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish 2023-05-15 17:32 ` [PATCH v3 1/2] scripts/tags.sh: " Ahmed S. Darwish 2023-05-15 17:32 ` [PATCH v3 2/2] docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 Ahmed S. Darwish @ 2023-05-20 22:41 ` Masahiro Yamada 2 siblings, 0 replies; 17+ messages in thread From: Masahiro Yamada @ 2023-05-20 22:41 UTC (permalink / raw) To: Ahmed S. Darwish Cc: Nathan Chancellor, Nick Desaulniers, Nicolas Schier, Jonathan Corbet, Thomas Gleixner, linux-kbuild, linux-doc, LKML On Tue, May 16, 2023 at 2:33 AM Ahmed S. Darwish <darwi@linutronix.de> wrote: > > Hi, > > v3-changelog > ------------ > > Handle review remarks from Masahiro Yamada: > > - Apply shellcheck on new "scripts/tags.sh" code. > > - Shorten code through shell's "default value" parameter expansion. > > NEW: > > - Cc docs maintainer (Documentation/process/changes.rst change). > > Thanks! > > v2-changelog > ------------ > > https://lkml.kernel.org/r/20230509012616.81579-1-darwi@linutronix.de > > Handle review remarks from Masahiro Yamada: > > - scripts/tags.sh: remove the O= language, and focus on the general > case of the build directory being different from the kernel source > tree, as specified in kernel Makefile L159. > > - Fix failure when build directory is a subdirectory of the kernel > source tree. > > NEW: > > - Update Documentation/process/changes.rst with new gtags (GNU GLOBAL) > requirements. > > Thanks! > > Cover letter / v1 > ----------------- > > https://lkml.kernel.org/r/20230504201833.202494-1-darwi@linutronix.de > > make gtags for O= kernel builds is currently broken. For example, when doing: > > make O=../build/ x86_64_defconfig > make O=../build/ gtags > > gtags generates a warning for each kernel source file to be indexed: > > make[1]: Entering directory '/home/darwi/build' > GEN gtags > Warning: '/home/darwi/linux/arch/x86/include/asm/qspinlock.h' is out of source tree. ignored. > Warning: '/home/darwi/linux/arch/x86/include/asm/hpet.h' is out of source tree. ignored. > ... > Warning: '/home/darwi/linux/virt/lib/irqbypass.c' is out of source tree. ignored. > make[1]: Leaving directory '/home/darwi/build/' > > and then generates an empty index: > > $ du -hs ~/build/G* > 16K /home/darwi/build/GPATH > 16K /home/darwi/build/GRTAGS > 16K /home/darwi/build/GTAGS > > This series includes a proposed fix. After applying it: > > $ make O=../build/ gtags > make[1]: Entering directory '/home/darwi/build' > GEN gtags > make[1]: Leaving directory '/home/darwi/build' > > $ du -hs ~/build/G* > 9.1M /home/darwi/build/GPATH > 506M /home/darwi/build/GRTAGS > 696M /home/darwi/build/GTAGS > > The generated files can then be integrated with editors or IDEs as > usual. > > => > > Ahmed S. Darwish (2): > scripts/tags.sh: Resolve gtags empty index generation > docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 Both applied. Thanks. > > Documentation/process/changes.rst | 7 +++++++ > scripts/tags.sh | 9 ++++++++- > 2 files changed, 15 insertions(+), 1 deletion(-) > > base-commit: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6 > -- > 2.30.2 -- Best Regards Masahiro Yamada ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2023-05-20 22:41 UTC | newest] Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-05-04 20:18 [PATCH v1 0/1] scripts: Fix "make gtags" for O= kernel builds Ahmed S. Darwish 2023-05-04 20:18 ` [PATCH v1 1/1] scripts/tags.sh: Fix gtags generation " Ahmed S. Darwish 2023-05-04 21:32 ` Nathan Chancellor 2023-05-04 22:00 ` Ahmed S. Darwish 2023-05-05 5:13 ` Masahiro Yamada 2023-05-05 5:17 ` Masahiro Yamada 2023-05-08 14:11 ` Ahmed S. Darwish 2023-05-09 1:26 ` [PATCH v2 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish 2023-05-09 1:26 ` [PATCH v2 1/2] scripts/tags.sh: " Ahmed S. Darwish 2023-05-11 18:51 ` Masahiro Yamada 2023-05-15 15:23 ` Ahmed S. Darwish 2023-05-15 16:35 ` Ahmed S. Darwish 2023-05-09 1:26 ` [PATCH v2 2/2] docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 Ahmed S. Darwish 2023-05-15 17:32 ` [PATCH v3 0/2] scripts: Resolve gtags empty index generation Ahmed S. Darwish 2023-05-15 17:32 ` [PATCH v3 1/2] scripts/tags.sh: " Ahmed S. Darwish 2023-05-15 17:32 ` [PATCH v3 2/2] docs: Set minimal gtags / GNU GLOBAL version to 6.6.5 Ahmed S. Darwish 2023-05-20 22:41 ` [PATCH v3 0/2] scripts: Resolve gtags empty index generation Masahiro Yamada
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).