clang static analyzer

clang static analyzer

[Tomas Carnecky]

There have been several attempts at running the clang static analyzer on the git source code, some even resulted in patches. I tried it, too, and among the many false positives I think clang found a few real issues. The results can be seen at [1].

Clang again found many dead assignments/increments, but in the earlier discussions you concluded that you want to keep those around. So I focussed on another class of potential bugs: Argument with 'nonnull' attribute passed null. There were a total of seven such issues. I then tried to look through the code and see if they are valid or false positives:

xdiff-interface.c:xdiff_set_find_func() - When 'value' is a string with no newline character in it, the loop at line 291 sets 'value' to NULL on its first iteration and then passes 'value' to strchr() in the second iteration.

utf8.c:utf8_strwidth() - 'string' may be set to NULL in utf8_width() which makes this one a false positive.

pretty.c:get_header() - if 'line' doesn't contain a newline character, line is set to NULL on first iteration and then passed to strchr() in the second itration.

attr.c:prepare_attr_stack() - bootstrap_attr_stack() sets attr_stack so this one is a false positive as well.

test-parse-options.c:length_callback() - if arg == NULL and unset == 0 then the function passes NULL to strlen().

builtin-pack-objects.c:check_pbase_path() - false positive, if done_pbase_paths == NULL then also done_pbase_paths_alloc == 0 and so step 4 can't take the false branch.

builtin-ls-files.c:verify_pathspec() - false positive, pathspec is not NULL when the function is called.


- Some of the issues might be purely hypothetical, for example I don't know if it's possible that get_header() can be passed a string with no newlines, maybe this is prevented earlier in the code path.
- Some of the false positives (such as the last one) could be avoided by giving clang a hint that a certain variable can't be NULL (by using assert() or if (!foo) return).


tom


[1] http://78.46.209.101/stuff/clang-static-analyzer/git/v1.6.6-rc1-32-g97f3d79/

Re: clang static analyzer

[Jeff King]

On Sun, Dec 06, 2009 at 07:11:24AM +0100, Tomas Carnecky wrote:

> Clang again found many dead assignments/increments, but in the earlier
> discussions you concluded that you want to keep those around. So I
> focussed on another class of potential bugs: Argument with 'nonnull'
> attribute passed null. There were a total of seven such issues. I then
> tried to look through the code and see if they are valid or false
> positives:

Thanks, I think you are moving in the right direction to manually
investigate the output of clang, since it obviously does generate some
false positives.

I think the next step for each site you found would be:

  1. If it really is a problem, then it should be easy to show a simple
     case that can trigger the issue. Submit a patch fixing that site,
     either describing the test case in the commit message, or adding a
     case to the test suite.

  2. If it is a false positive, see what it would take to silence clang
     and submit a patch.  I don't think we are opposed to annotations
     that help analysis tools as long as those annotations aren't too
     intrusive or make the code less readable.

-Peff

Re: clang static analyzer

[Nicolas Pitre]

On Sun, 6 Dec 2009, Jeff King wrote:

> On Sun, Dec 06, 2009 at 07:11:24AM +0100, Tomas Carnecky wrote:
> 
> > Clang again found many dead assignments/increments, but in the earlier
> > discussions you concluded that you want to keep those around. So I
> > focussed on another class of potential bugs: Argument with 'nonnull'
> > attribute passed null. There were a total of seven such issues. I then
> > tried to look through the code and see if they are valid or false
> > positives:
> 
> Thanks, I think you are moving in the right direction to manually
> investigate the output of clang, since it obviously does generate some
> false positives.
> 
> I think the next step for each site you found would be:
> 
>   1. If it really is a problem, then it should be easy to show a simple
>      case that can trigger the issue. Submit a patch fixing that site,
>      either describing the test case in the commit message, or adding a
>      case to the test suite.
> 
>   2. If it is a false positive, see what it would take to silence clang
>      and submit a patch.  I don't think we are opposed to annotations
>      that help analysis tools as long as those annotations aren't too
>      intrusive or make the code less readable.

I'm a bit skeptical here.  Going down that route might mean that we'll 
eventually have to add all sort of crap to accommodate everyone's 
preferred static analysis tool of the day.  Would be far nicer to try to 
make those tools more intelligent instead, or at least make them 
understand an out-of-line annotation format that does not clutter the 
code itself.


Nicolas

Re: clang static analyzer

[Jeff King]

On Sun, Dec 06, 2009 at 10:39:56AM -0500, Nicolas Pitre wrote:

> >   2. If it is a false positive, see what it would take to silence clang
> >      and submit a patch.  I don't think we are opposed to annotations
> >      that help analysis tools as long as those annotations aren't too
> >      intrusive or make the code less readable.
> 
> I'm a bit skeptical here.  Going down that route might mean that we'll 
> eventually have to add all sort of crap to accommodate everyone's 
> preferred static analysis tool of the day.  Would be far nicer to try to 
> make those tools more intelligent instead, or at least make them 
> understand an out-of-line annotation format that does not clutter the 
> code itself.

To be clear, I am a bit skeptical, too. I would really prefer an
out-of-line annotation if one is available. But I am trying to encourage
the OP to actually make a patch for one instance so we can see just what
it would look like. Then we actually have a data point to discuss.

-Peff

Re: clang static analyzer

[Nicolas Sebrecht]

The 06/12/09, Jeff King wrote:
> On Sun, Dec 06, 2009 at 10:39:56AM -0500, Nicolas Pitre wrote:
> 
> > >   2. If it is a false positive, see what it would take to silence clang
> > >      and submit a patch.  I don't think we are opposed to annotations
> > >      that help analysis tools as long as those annotations aren't too
> > >      intrusive or make the code less readable.
> > 
> > I'm a bit skeptical here.  Going down that route might mean that we'll 
> > eventually have to add all sort of crap to accommodate everyone's 
> > preferred static analysis tool of the day.  Would be far nicer to try to 
> > make those tools more intelligent instead, or at least make them 
> > understand an out-of-line annotation format that does not clutter the 
> > code itself.
> 
> To be clear, I am a bit skeptical, too. 

Me too. I have no idea about how clang works but if there are enough
work done to support such a tool in the code itself, it would be sad to
not share and promote this work. If Junio cares enough himself, he could
set up a public dedicated branch. Now if he doesn't, it's not necessary
at all. Tomas or anyone else with enough time and motivation can fork
the repository for this purpose.

The latter option would be good and appreciated by everybody here, I
think.

-- 
Nicolas Sebrecht

Re: clang static analyzer

[Junio C Hamano]

Tomas Carnecky <tom@dbservice.com> writes:

> pretty.c:get_header() - if 'line' doesn't contain a newline character,
> line is set to NULL on first iteration and then passed to strchr() in
> the second itration.

Thanks.

In practice, we will always have a newline, as we are reading from a valid
commit object in this codepath.


 pretty.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/pretty.c b/pretty.c
index 8f5bd1a..7eb5384 100644
--- a/pretty.c
+++ b/pretty.c
@@ -245,11 +245,11 @@ static char *get_header(const struct commit *commit, const char *key)
 	int key_len = strlen(key);
 	const char *line = commit->buffer;
 
-	for (;;) {
+	while (line) {
 		const char *eol = strchr(line, '\n'), *next;
 
 		if (line == eol)
-			return NULL;
+			break;
 		if (!eol) {
 			eol = line + strlen(line);
 			next = NULL;
@@ -262,6 +262,7 @@ static char *get_header(const struct commit *commit, const char *key)
 		}
 		line = next;
 	}
+	return NULL;
 }
 
 static char *replace_encoding_header(char *buf, const char *encoding)

Re: clang static analyzer

[Junio C Hamano]

Tomas Carnecky <tom@dbservice.com> writes:

> xdiff-interface.c:xdiff_set_find_func() - When 'value' is a string with
> no newline character in it, the loop at line 291 sets 'value' to NULL on
> its first iteration and then passes 'value' to strchr() in the second
> iteration.

Thanks, but I am confused with your analysis.

If value doesn't have '\n', then regs->nr is 1 when you go into the loop
at ll. 291-, because we counted the number of LF in the first loop in the
function.

The first iteration of the second loop sets ep to NULL, expression is set
to value, then we run regcomp on the expression.  Then at the end of the
loop we do set value to a bogus "(char*)1".  But incrementing i makes it
go over regs->nr and satisfy the termination condition of the loop; we
happily exit the loop before we use the now bogus "value".