* [PATCH v3] gen_compile_commands: handle multiple lines per .mod file
@ 2022-06-28 1:23 John Hubbard
2022-06-28 18:45 ` Nick Desaulniers
0 siblings, 1 reply; 3+ messages in thread
From: John Hubbard @ 2022-06-28 1:23 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Nick Desaulniers, Nathan Chancellor, Tom Rix, Jason Gunthorpe,
LKML, clang-built-linux, John Hubbard
scripts/clang-tools/gen_compile_commands.py incorrectly assumes that
each .mod file only contains one line. That assumption was correct when
the script was originally created, but commit 9413e7640564 ("kbuild:
split the second line of *.mod into *.usyms") changed the .mod file
format so that there is one entry per line, and potentially many lines.
The problem can be reproduced by using Kbuild to generate
compile_commands.json, like this:
make CC=clang compile_commands.json
In many cases, the problem might be overlooked because many subsystems
only have one line anyway. However, in some subsystems (Nouveau, with
762 entries, is a notable example) it results in skipping most of the
subsystem.
Fix this by fully processing each .mod file.
Fixes: 9413e7640564 ("kbuild: split the second line of *.mod into *.usyms")
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
Link to v2:
https://lore.kernel.org/all/20220616021149.327587-1-jhubbard@nvidia.com/
Changes since v2:
* Restored a blank line.
* Rewrote the commit description.
thanks,
John Hubbard
scripts/clang-tools/gen_compile_commands.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py
index 1d1bde1fd45e..47da25b3ba7d 100755
--- a/scripts/clang-tools/gen_compile_commands.py
+++ b/scripts/clang-tools/gen_compile_commands.py
@@ -157,10 +157,10 @@ def cmdfiles_for_modorder(modorder):
if ext != '.ko':
sys.exit('{}: module path must end with .ko'.format(ko))
mod = base + '.mod'
- # The first line of *.mod lists the objects that compose the module.
+ # Read from *.mod, to get a list of objects that compose the module.
with open(mod) as m:
- for obj in m.readline().split():
- yield to_cmdfile(obj)
+ for mod_line in m:
+ yield to_cmdfile(mod_line.rstrip())
def process_line(root_directory, command_prefix, file_path):
--
2.36.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v3] gen_compile_commands: handle multiple lines per .mod file
2022-06-28 1:23 [PATCH v3] gen_compile_commands: handle multiple lines per .mod file John Hubbard
@ 2022-06-28 18:45 ` Nick Desaulniers
2022-06-29 2:45 ` Masahiro Yamada
0 siblings, 1 reply; 3+ messages in thread
From: Nick Desaulniers @ 2022-06-28 18:45 UTC (permalink / raw)
To: John Hubbard
Cc: Masahiro Yamada, Nathan Chancellor, Tom Rix, Jason Gunthorpe,
LKML, clang-built-linux
On Mon, Jun 27, 2022 at 6:24 PM John Hubbard <jhubbard@nvidia.com> wrote:
>
> scripts/clang-tools/gen_compile_commands.py incorrectly assumes that
> each .mod file only contains one line. That assumption was correct when
> the script was originally created, but commit 9413e7640564 ("kbuild:
> split the second line of *.mod into *.usyms") changed the .mod file
> format so that there is one entry per line, and potentially many lines.
>
> The problem can be reproduced by using Kbuild to generate
> compile_commands.json, like this:
>
> make CC=clang compile_commands.json
>
> In many cases, the problem might be overlooked because many subsystems
> only have one line anyway. However, in some subsystems (Nouveau, with
> 762 entries, is a notable example) it results in skipping most of the
> subsystem.
>
> Fix this by fully processing each .mod file.
>
> Fixes: 9413e7640564 ("kbuild: split the second line of *.mod into *.usyms")
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Cc: Nick Desaulniers <ndesaulniers@google.com>
Thanks for the fix, John!
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
> ---
>
> Link to v2:
> https://lore.kernel.org/all/20220616021149.327587-1-jhubbard@nvidia.com/
>
> Changes since v2:
>
> * Restored a blank line.
>
> * Rewrote the commit description.
>
> thanks,
> John Hubbard
>
>
> scripts/clang-tools/gen_compile_commands.py | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py
> index 1d1bde1fd45e..47da25b3ba7d 100755
> --- a/scripts/clang-tools/gen_compile_commands.py
> +++ b/scripts/clang-tools/gen_compile_commands.py
> @@ -157,10 +157,10 @@ def cmdfiles_for_modorder(modorder):
> if ext != '.ko':
> sys.exit('{}: module path must end with .ko'.format(ko))
> mod = base + '.mod'
> - # The first line of *.mod lists the objects that compose the module.
> + # Read from *.mod, to get a list of objects that compose the module.
> with open(mod) as m:
> - for obj in m.readline().split():
> - yield to_cmdfile(obj)
> + for mod_line in m:
> + yield to_cmdfile(mod_line.rstrip())
>
>
> def process_line(root_directory, command_prefix, file_path):
> --
> 2.36.1
>
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v3] gen_compile_commands: handle multiple lines per .mod file
2022-06-28 18:45 ` Nick Desaulniers
@ 2022-06-29 2:45 ` Masahiro Yamada
0 siblings, 0 replies; 3+ messages in thread
From: Masahiro Yamada @ 2022-06-29 2:45 UTC (permalink / raw)
To: Nick Desaulniers
Cc: John Hubbard, Nathan Chancellor, Tom Rix, Jason Gunthorpe, LKML,
clang-built-linux
On Wed, Jun 29, 2022 at 3:45 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Mon, Jun 27, 2022 at 6:24 PM John Hubbard <jhubbard@nvidia.com> wrote:
> >
> > scripts/clang-tools/gen_compile_commands.py incorrectly assumes that
> > each .mod file only contains one line. That assumption was correct when
> > the script was originally created, but commit 9413e7640564 ("kbuild:
> > split the second line of *.mod into *.usyms") changed the .mod file
> > format so that there is one entry per line, and potentially many lines.
> >
> > The problem can be reproduced by using Kbuild to generate
> > compile_commands.json, like this:
> >
> > make CC=clang compile_commands.json
> >
> > In many cases, the problem might be overlooked because many subsystems
> > only have one line anyway. However, in some subsystems (Nouveau, with
> > 762 entries, is a notable example) it results in skipping most of the
> > subsystem.
> >
> > Fix this by fully processing each .mod file.
> >
> > Fixes: 9413e7640564 ("kbuild: split the second line of *.mod into *.usyms")
> > Cc: Masahiro Yamada <masahiroy@kernel.org>
> > Cc: Nick Desaulniers <ndesaulniers@google.com>
>
> Thanks for the fix, John!
> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Applied to linux-kbuild/fixes with Nick's reviewed-by.
Thanks.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-06-29 3:00 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-28 1:23 [PATCH v3] gen_compile_commands: handle multiple lines per .mod file John Hubbard
2022-06-28 18:45 ` Nick Desaulniers
2022-06-29 2:45 ` Masahiro Yamada
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.