Re: [cocci] [PATCH] Documentation: dev-tools: Add a section for static analysis tools

[Julia Lawall <julia.lawall@inria.fr>]

On Tue, 29 Mar 2022, Dan Carpenter wrote:

> On Mon, Mar 28, 2022 at 05:37:28PM -0500, Daniel Latypov wrote:
> > On Mon, Mar 28, 2022 at 4:20 PM Marcelo Schmitt
> > <marcelo.schmitt1@gmail.com> wrote:
> > >
> > > Complement the Kernel Testing Guide documentation page by adding a
> > > section about static analysis tools.
> > >
> > > Signed-off-by: Marcelo Schmitt <marcelo.schmitt1@gmail.com>
> >
> >
> > > ---
> > > Hey everyone,
> > >
> > > I think this patch can be a good addition to the documentation as
> > > discussed in the thread for the testing guide documentation page:
> > > Link: https://lore.kernel.org/linux-doc/CABVgOS=2iYtqTVdxwH=mcFpcSuLP4cpJ4s6PKP4Gc-SH6jidgQ@mail.gmail.com/
> > >
> > > If you think it would be worth it, I can try making something more
> >
> > Nice!
> > This is definitely worth including, but I'm out of my depth here.
> > Some ideas below in case they're helpful.
> >
> > Acked-by: Daniel Latypov <dlatypov@google.com>
> >
> > > elaborated. Maybe provide some guidance on when to use each tool.
> > > I've been studying how Linux device drivers are tested.
> > > Here's a post I wrote talking about some testing tools.
> > > Link: https://marcelosc.gitlab.io/how-is-linux-tested/
> > >
> > > Best regards,
> > > Marcelo
> > >
> > >  Documentation/dev-tools/testing-overview.rst | 29 ++++++++++++++++++++
> > >  1 file changed, 29 insertions(+)
> > >
> > > diff --git a/Documentation/dev-tools/testing-overview.rst b/Documentation/dev-tools/testing-overview.rst
> > > index 65feb81edb14..b00511109a9d 100644
> > > --- a/Documentation/dev-tools/testing-overview.rst
> > > +++ b/Documentation/dev-tools/testing-overview.rst
> > > @@ -115,3 +115,32 @@ that none of these errors are occurring during the test.
> > >  Some of these tools integrate with KUnit or kselftest and will
> > >  automatically fail tests if an issue is detected.
> > >
> > > +Static Analysis Tools
> > > +======================
> > > +
> > > +In addition to testing a running kernel, one may also scout for bugs by
> > > +analyzing the source code semantics. Three tools are well known for serving this
> > > +purpose.
> > > +
> > > +Sparse can help test the kernel by performing type-checking, lock checking,
> > > +value range checking, in addition to reporting various errors and warnings while
> > > +examining the code. See the Documentation/dev-tools/sparse.rst documentation
> > > +page for details on how to use it.
> > > +
> > > +Smatch extends Sparse and provides additional checks for programming logic
> >
> > Coming from a place of ignorance, when should I use sparse vs smatch?
> > Is there an existing consensus on this, or is that a controversial question?
>
> I am the author of Smatch.
>
> Sparse is a compiler which can print static checker warnings.  Smatch
> uses Sparse as a C front end.
>
> Sparse is useful for type checking, detecting places which use __user
> pointers improperly or endian bugs.  Sparse is much faster than Smatch.
>
> Smatch does flow analysis and if you build the DB, then it does
> cross function analysis.  Smatch tries to answer questions like where is
> this buffer allocated?  How big is it?  Can this index be controlled by
> the user?  Is this variable larger than that variable?  I feel like it's
> generally easier to write checks in Smatch than it is to write checks in
> Sparse.
>
> Coccinelle is probably the easiest for writing checks.  It works before
> the pre-compiler so it easier to check for bugs in macros using
> Coccinelle.  Coccinelle also writes patches fixes for you which no one
> else does.
>
> There are sometimes overlap in their checks.  But I'm not going to
> re-implement Sparse's check in Smatch because Sparse already does that
> well.  If it's just way easier to write the check with Coccinelle then
> do that.
>
> With Coccinelle you can do a mass conversion from
> kmalloc(x * size, GFP_KERNEL) to kmalloc_array(x, size, GFP_KERNEL); and
> that's really useful.  If you just created a Smatch warning and try to
> push the work of converting on to the maintainers they would be annoyed.
> You'd have to argue about each warning if can really overflow or not.

Coccinelle does no analysis of variable values, which is the strong point
of smatch.  On the other hand, as Dan points out, it allows you to do
simple things in a simple way.

julia