[PATCH] sparse: Add CLOG option for saving warning

[PATCH] sparse: Add CLOG option for saving warning

[Christopher Li]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

Currently sparse warning only output to stderr.
In the parallel build process, different source
file output is fixed together. It is hard to keep
track of the warning.

Add the CLOG=<logname> option in command line to
save the sparse warning into individual log file.

Typical usage:

make -j8 C=2 CLOG=<logname>

The log file is saved in the target directory as
.xxx.o.<logname>.sparse

By diffing between different log file, it is much
easier to analyze how the sparse change impact
the whole kernel build.

Signed-off-by: Christopher Li <sparse@chrisli.org>


Chris

[-- Attachment #2: 0001-sparse-Add-CLOG-option-for-saving-warning.patch --]
[-- Type: text/x-patch, Size: 3006 bytes --]

From 3b2ff204cbda684adf9dba2adf568062533ae34d Mon Sep 17 00:00:00 2001
From: Christopher Li <sparse@chrisli.org>
Date: Mon, 30 Jun 2014 01:33:22 -0700
Subject: [PATCH] sparse: Add CLOG option for saving warning

Currently sparse warning only output to stderr.
In the parallel build process, different source
file output is fixed together. It is hard to keep
track of the warning.

Add the CLOG=<logname> option in command line to
save the sparse warning into individual log file.

Typical usage:

make -j8 C=2 CLOG=<logname>

The log file is saved in the target directory as
.xxx.o.<logname>.sparse

By diffing between different log file, it is much
easier to analyze how the sparse change impact
the whole kernel build.

Signed-off-by: Christopher Li <sparse@chrisli.org>
---
 Makefile               | 13 ++++++++++++-
 scripts/Makefile.build | 11 +++++++++--
 2 files changed, 21 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index b11e2d5..56c3502 100644
--- a/Makefile
+++ b/Makefile
@@ -68,6 +68,16 @@ ifndef KBUILD_CHECKSRC
   KBUILD_CHECKSRC = 0
 endif
 
+ifeq ("$(origin CLOG)", "command line")
+  KBUILD_CHECKLOG = $(CLOG)
+endif
+ifndef KBUILD_CHECKLOG
+  KBUILD_CHECKLOG =
+endif
+
+
+
+
 # Use make M=dir to specify directory of external module to build
 # Old syntax make ... SUBDIRS=$PWD is still supported
 # Setting the environment variable KBUILD_EXTMOD take precedence
@@ -287,7 +297,7 @@ ifeq ($(MAKECMDGOALS),)
 endif
 
 export KBUILD_MODULES KBUILD_BUILTIN
-export KBUILD_CHECKSRC KBUILD_SRC KBUILD_EXTMOD
+export KBUILD_CHECKSRC KBUILD_CHECKLOG KBUILD_SRC KBUILD_EXTMOD
 
 # Beautify output
 # ---------------------------------------------------------------------------
@@ -1370,6 +1380,7 @@ clean: $(clean-dirs)
 	$(call cmd,rmfiles)
 	@find $(if $(KBUILD_EXTMOD), $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \
 		\( -name '*.[oas]' -o -name '*.ko' -o -name '.*.cmd' \
+		-o -name '.*.sparse' \
 		-o -name '*.ko.*' \
 		-o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
 		-o -name '*.symtypes' -o -name 'modules.order' \
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index bf3e677..45c6004 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -96,14 +96,21 @@ __build: $(if $(KBUILD_BUILTIN),$(builtin-target) $(lib-target) $(extra-y)) \
 	 $(subdir-ym) $(always)
 	@:
 
+check_log_file = $(dot-target).$(KBUILD_CHECKLOG).sparse
+
+ifneq ($(KBUILD_CHECKLOG),)
+  check_logging = 2> $(check_log_file)
+endif
+
+cmd_check = $(CHECK) $(CHECKFLAGS) $(c_flags) $< $(check_logging) ;
 # Linus' kernel sanity checking tool
 ifneq ($(KBUILD_CHECKSRC),0)
   ifeq ($(KBUILD_CHECKSRC),2)
     quiet_cmd_force_checksrc = CHECK   $<
-          cmd_force_checksrc = $(CHECK) $(CHECKFLAGS) $(c_flags) $< ;
+          cmd_force_checksrc = $(cmd_check)
   else
       quiet_cmd_checksrc     = CHECK   $<
-            cmd_checksrc     = $(CHECK) $(CHECKFLAGS) $(c_flags) $< ;
+            cmd_checksrc     = $(cmd_check)
   endif
 endif
 
-- 
1.9.3

Re: [PATCH] sparse: Add CLOG option for saving warning

[Christopher Li]

On Mon, Jun 30, 2014 at 1:57 AM, Christopher Li <sparse@chrisli.org> wrote:
>
> Add the CLOG=<logname> option in command line to
> save the sparse warning into individual log file.
>
> Typical usage:
>
> make -j8 C=2 CLOG=<logname>

Any feed back for this change? I want to clarify that this patch
is for the Linux kernel kbuild system, not sparse.

Thanks

Chris

Re: [PATCH] sparse: Add CLOG option for saving warning

[Josh Triplett]

On Fri, Jul 04, 2014 at 12:48:00AM -0700, Christopher Li wrote:
> On Mon, Jun 30, 2014 at 1:57 AM, Christopher Li <sparse@chrisli.org> wrote:
> >
> > Add the CLOG=<logname> option in command line to
> > save the sparse warning into individual log file.
> >
> > Typical usage:
> >
> > make -j8 C=2 CLOG=<logname>
> 
> Any feed back for this change? I want to clarify that this patch
> is for the Linux kernel kbuild system, not sparse.

Seems reasonable to me.

- Josh Triplett

Re: [PATCH] sparse: Add CLOG option for saving warning

[Sam Ravnborg]

On Fri, Jul 04, 2014 at 12:48:00AM -0700, Christopher Li wrote:
> On Mon, Jun 30, 2014 at 1:57 AM, Christopher Li <sparse@chrisli.org> wrote:
> >
> > Add the CLOG=<logname> option in command line to
> > save the sparse warning into individual log file.
> >
> > Typical usage:
> >
> > make -j8 C=2 CLOG=<logname>
> 
> Any feed back for this change? I want to clarify that this patch
> is for the Linux kernel kbuild system, not sparse.
We do not need this kind of special handling of outputs from gcc.
For sparse you just do a run with C=2 then you have it.

In other words - this looks like overkill for somethign thas is achievable
with simpler means.

	Sam

Re: [PATCH] sparse: Add CLOG option for saving warning

[Christopher Li]

On Mon, Jul 7, 2014 at 4:22 AM, Sam Ravnborg <sam@ravnborg.org> wrote:

>> > Typical usage:
>> >
>> > make -j8 C=2 CLOG=<logname>
>>
> We do not need this kind of special handling of outputs from gcc.
> For sparse you just do a run with C=2 then you have it.
>
> In other words - this looks like overkill for somethign thas is achievable
> with simpler means.

I am aware of the C=2 flag. However, it does not provide consistent result
with "make -jn" flag.

Please consider that the gcc case is different. The kernel source is usually
clean of gcc warnings. If there is an error with gcc, the build process
stops. It is different with sparse. The primary goal of running sparse in
kernel build is to see those warning. Also sparse is a lot noisier than gcc
in the kernel build, so there is a need with logging which is not present
with gcc.

With this patch, here is a normal test procedure for me to see the impact of a
sparse change on kernel build:

$ make -j8 C=2 CLOG=std

# apply sparse change and make sparse

$ make -j8 C=2 CLOG=std-exp

$ find -name ".*.std.sparse" | while read -r file; do diff -du $file
${file/std.sparse/std-exp.
sparse} ; done > /tmp/sparse-diff


Without the CLOG= option, the only way to get similar diff
result is disable "-jn" option. Which make the test process painfully
slow.

I agree if I don't use "make -jn" flags, using C=2 is good enough.
However, what is your suggestion if I do want to use "make -jn" for
sparse checking in the previous mentioned usage case?

Thanks

Chris

Re: [PATCH] sparse: Add CLOG option for saving warning

[Dan Carpenter]

My kernel tree is full of drivers/foo.c.smatch and
drivers/foo.c.smatch-info files...

It would be nice to add it to .gitignore as well.

regards,
dan carpenter

Re: [PATCH] sparse: Add CLOG option for saving warning

[Christopher Li]

On Tue, Jul 8, 2014 at 12:37 AM, Dan Carpenter <dan.carpenter@oracle.com> wrote:
> My kernel tree is full of drivers/foo.c.smatch and
> drivers/foo.c.smatch-info files...
>
> It would be nice to add it to .gitignore as well.

Actually, ".*" is already in the .gitignore, there for the sparse log files
are covered.

Chris