Re: [PATCH v6] coccinelle: semantic code search for missing put_device()

Re: [PATCH v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

On Sat, 16 Feb 2019, Wen Yang wrote:

> The of_find_device_by_node() takes a reference to the underlying device
> structure, we should release that reference.
> The implementation of this semantic code search is:
> In a function, for a local variable obtained by of_find_device_by_node(),
> a, if it is released by a function such as
>    put_device()/of_dev_put()/platform_device_put() after the last use,
>    it is considered that there is no reference leak;
> b, if it is passed back to the caller via
>    dev_get_drvdata()/platform_get_drvdata()/get_device(), etc., the
>    reference will be released in other functions, and the current function
>    also considers that there is no reference leak;
> c, for the rest of the situation, the current function should release the
>    reference by calling put_device, this code search will report an error
>    with a specific confidence.
>
> By using this semantic code search, we have found some issues, such as:
> commit 11907e9d3533 ("ASoC: fsl-asoc-card: fix object reference leaks in
> fsl_asoc_card_probe")
> commit a12085d13997 ("mtd: rawnand: atmel: fix possible object reference
> leak")
> commit 11493f26856a ("mtd: rawnand: jz4780: fix possible object reference
> leak")
>
> There are still dozens of reference leaks in the current kernel code.
>
> Further, for the case of b, the object returned to other functions may also
> have a reference leak, we will continue to develop other cocci scripts to
> further check the reference leak.
>
> Signed-off-by: Wen Yang <yellowriver2010@hotmail.com>
> Reviewed-by: Julia Lawall <Julia.Lawall@lip6.fr>

Acked-by: Julia Lawall <julia.lawall@lip6.fr>

> Reviewed-by: Markus Elfring <Markus.Elfring@web.de>
> Cc: Julia Lawall <Julia.Lawall@lip6.fr>
> Cc: Gilles Muller <Gilles.Muller@lip6.fr>
> Cc: Nicolas Palix <nicolas.palix@imag.fr>
> Cc: Michal Marek <michal.lkml@markovi.net>
> Cc: Markus Elfring <Markus.Elfring@web.de>
> Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
> Cc: Wen Yang <wen.yang99@zte.com.cn>
> Cc: cheng.shengyu@zte.com.cn
> Cc: cocci@systeme.lip6.fr
> Cc: linux-kernel@vger.kernel.org
> ---
> v6:
> - to be double sure, replace &id->dev with (T)(&id->dev).
> - long string literals can be accepted because of error message search concerns around a tool like grep
> v5:
> - exchange the word patch by code search.
> - add a SPDX identifier.
> - a split string literal can be unwanted.
> - Change the content of the reported information.
> v4:
> - add Masahiro Yamada
> - omit a blank line
> - split the long message parameter
> - reduce the number of metavariables
> - Describe the implementation of the semantic patch,
>   explain the scenarios it can detect,
>   and further software development considerations.
> v3:
> - reduction of a bit of redundant C code within SmPL search specifications.
> - consider the message construction without using the extra Python variable msg.
> v2:
> - put exists after search, and then drop the when exists below.
> - should not use the same e as in the when's below.
> - Make a new type metavariable and use it to put a cast on the result of platform_get_drvdata.
>
>  scripts/coccinelle/free/put_device.cocci | 55 ++++++++++++++++++++++++
>  1 file changed, 55 insertions(+)
>  create mode 100644 scripts/coccinelle/free/put_device.cocci
>
> diff --git a/scripts/coccinelle/free/put_device.cocci b/scripts/coccinelle/free/put_device.cocci
> new file mode 100644
> index 000000000000..96e2508c0be1
> --- /dev/null
> +++ b/scripts/coccinelle/free/put_device.cocci
> @@ -0,0 +1,55 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/// Find missing put_device for every of_find_device_by_node.
> +///
> +// Confidence: Moderate
> +// Copyright: (C) 2018-2019 Wen Yang, ZTE.
> +// Comments:
> +// Options: --no-includes --include-headers
> +
> +virtual report
> +virtual org
> +
> +@search exists@
> +local idexpression id;
> +expression x,e,e1;
> +position p1,p2;
> +type T,T1,T2;
> +@@
> +
> +id = of_find_device_by_node@p1(x)
> +... when != e = id
> +if (id == NULL || ...) { ... return ...; }
> +... when != put_device(&id->dev)
> +    when != platform_device_put(id)
> +    when != of_dev_put(id)
> +    when != if (id) { ... put_device(&id->dev) ... }
> +    when != e1 = (T)id
> +    when != e1 = (T)(&id->dev)
> +    when != e1 = get_device(&id->dev)
> +    when != e1 = (T)platform_get_drvdata(id)
> +(
> +  return
> +(    id
> +|    (T1)dev_get_drvdata(&id->dev)
> +|    (T2)platform_get_drvdata(id)
> +);
> +| return@p2 ...;
> +)
> +
> +@script:python depends on report@
> +p1 << search.p1;
> +p2 << search.p2;
> +@@
> +
> +coccilib.report.print_report(p2[0],
> +			     "ERROR: missing put_device; call of_find_device_by_node on line "
> +                             + p1[0].line
> +                             + ", but without a corresponding object release within this function.")
> +
> +@script:python depends on org@
> +p1 << search.p1;
> +p2 << search.p2;
> +@@
> +
> +cocci.print_main("of_find_device_by_node", p1)
> +cocci.print_secs("needed put_device", p2)
> --
> 2.20.1
>
>

Re: [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

> In a function, for a local variable obtained by of_find_device_by_node(),

I got a software understanding where such a variable can not be obtained
from this function call.
The return value (like a pointer in this use case) can be stored there.


> v6:
> - to be double sure, replace &id->dev with (T)(&id->dev).

The support for data type casts is another interesting extension for
this source code analysis approach.
Further adjustments might become possible at other places of the presented SmPL script
after specific clarifications of previously mentioned implementation details.

Regards,
Markus

Re: 答复: [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

> But please also refer to the examples of coccinelle, such as:
> http://coccinelle.lip6.fr/rules/kmalloc.html
> and
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/coccinelle/free/pci_free_consistent.cocci

These scripts for the semantic patch language show some software design possibilities.
They contain implementation details which can be also worth for additional
development considerations.
Will systematic refactoring become more interesting?


> You will find that there are differences between coccinelle and c.

Would you like to discuss any of them further?

Regards,
Markus

Re: [PATCH v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

> +@search exists@
> +local idexpression id;
> +expression x,e,e1;
> +position p1,p2;
> +type T,T1,T2;
> +@@
> +
> +id = of_find_device_by_node@p1(x)
> +... when != e = id

I suggest to increase your software development attention also for
another implementation detail.
Source code analysis triggers challenges for safe data flow handling.
the semantic patch language supports search specifications for
the exclusion of specific assignments.

Does this SmPL code contain a questionable order for the source
and target metavariables?
Can the following variant be more appropriate?

+ ... when != id = e


> +if (id == NULL || ...) { ... return ...; }
> +... when != put_device(&id->dev)
> +    when != platform_device_put(id)
> +    when != of_dev_put(id)
> +    when != if (id) { ... put_device(&id->dev) ... }
> +    when != e1 = (T)id

Would you like to avoid that the return value from the shown function call
gets overwritten in the variable before it was used once at least
(when a bit of extra C code is tolerated before a null pointer check)?

Regards,
Markus

Re: [PATCH v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

On Sun, 17 Feb 2019, Markus Elfring wrote:

> > +@search exists@
> > +local idexpression id;
> > +expression x,e,e1;
> > +position p1,p2;
> > +type T,T1,T2;
> > +@@
> > +
> > +id = of_find_device_by_node@p1(x)
> > +... when != e = id
>
> I suggest to increase your software development attention also for
> another implementation detail.
> Source code analysis triggers challenges for safe data flow handling.
> the semantic patch language supports search specifications for
> the exclusion of specific assignments.
>
> Does this SmPL code contain a questionable order for the source
> and target metavariables?
> Can the following variant be more appropriate?
>
> + ... when != id = e

This is possible, but I think unlikely.

>
>
> > +if (id == NULL || ...) { ... return ...; }
> > +... when != put_device(&id->dev)
> > +    when != platform_device_put(id)
> > +    when != of_dev_put(id)
> > +    when != if (id) { ... put_device(&id->dev) ... }
> > +    when != e1 = (T)id
>
> Would you like to avoid that the return value from the shown function call
> gets overwritten in the variable before it was used once at least
> (when a bit of extra C code is tolerated before a null pointer check)?

Indeed there should be a put then too, but again, it seems unlikely.

julia


>
> Regards,
> Markus
>

Re: [PATCH v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>>> +@search exists@
>>> +local idexpression id;
>>> +expression x,e,e1;
>>> +position p1,p2;
>>> +type T,T1,T2;
>>> +@@
>>> +
>>> +id = of_find_device_by_node@p1(x)
>>> +... when != e = id
>>
>> I suggest to increase your software development attention also for
>> another implementation detail.
>> Source code analysis triggers challenges for safe data flow handling.
>> the semantic patch language supports search specifications for
>> the exclusion of specific assignments.
>>
>> Does this SmPL code contain a questionable order for the source
>> and target metavariables?
>> Can the following variant be more appropriate?
>>
>> + ... when != id = e
>
> This is possible, but I think unlikely.

Would you dare to interpret my update suggestion (reordering of two identifiers)
as a required SmPL script correction?

Regards,
Markus

Re: [PATCH v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

On Sun, 17 Feb 2019, Markus Elfring wrote:

> >>> +@search exists@
> >>> +local idexpression id;
> >>> +expression x,e,e1;
> >>> +position p1,p2;
> >>> +type T,T1,T2;
> >>> +@@
> >>> +
> >>> +id = of_find_device_by_node@p1(x)
> >>> +... when != e = id
> >>
> >> I suggest to increase your software development attention also for
> >> another implementation detail.
> >> Source code analysis triggers challenges for safe data flow handling.
> >> the semantic patch language supports search specifications for
> >> the exclusion of specific assignments.
> >>
> >> Does this SmPL code contain a questionable order for the source
> >> and target metavariables?
> >> Can the following variant be more appropriate?
> >>
> >> + ... when != id = e
> >
> > This is possible, but I think unlikely.
>
> Would you dare to interpret my update suggestion (reordering of two identifiers)
> as a required SmPL script correction?

I didn't suggest to reorder anything.  Both are needed.

And, no I don't consider it to be a required suggestion.  In practice,
reassigning such a variable is very unlikely.

julia

Re: [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>> Would you dare to interpret my update suggestion (reordering of two identifiers)
>> as a required SmPL script correction?
>
> I didn't suggest to reorder anything.

This is obvious according to your acknowledgement for the sixth version
of this evolving SmPL script.


> Both are needed.

If you would insist on the specification of such an assignment exclusion
for a SmPL ellipsis:
Can we agree on a correct order?


> And, no I don't consider it to be a required suggestion.

Have we got a different view about an implementation detail at this place?


> In practice, reassigning such a variable is very unlikely.

This can be.

Regards,
Markus

Re: [v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

On Sun, 17 Feb 2019, Markus Elfring wrote:

> >> Would you dare to interpret my update suggestion (reordering of two identifiers)
> >> as a required SmPL script correction?
> >
> > I didn't suggest to reorder anything.
>
> This is obvious according to your acknowledgement for the sixth version
> of this evolving SmPL script.
>
>
> > Both are needed.
>
> If you would insist on the specification of such an assignment exclusion
> for a SmPL ellipsis:
> Can we agree on a correct order?

I don't get your point.  There is no correct order.  Each order expresses
something different.  The order that is currently in the semantic patch is
the one that is more likely in practice.

julia

>
>
> > And, no I don't consider it to be a required suggestion.
>
> Have we got a different view about an implementation detail at this place?
>
>
> > In practice, reassigning such a variable is very unlikely.
>
> This can be.
>
> Regards,
> Markus
>

Re: [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>> If you would insist on the specification of such an assignment exclusion
>> for a SmPL ellipsis:
>> Can we agree on a correct order?
>
> I don't get your point.

I propose to take another closer look at a bit of SmPL code.


> There is no correct order.

I have got an other software development view here.


> Each order expresses something different.

I agree to this information.


> The order that is currently in the semantic patch is the one
> that is more likely in practice.

Please check once more.

…
+@search exists@
+local idexpression id;
+expression x,e,e1;
+position p1,p2;
…
+@@
+
+id = of_find_device_by_node@p1(x)
+... when != e = id
…

Or:

…
+ ... when != id = e
…


Which SmPL specification will achieve the desired software behaviour?

Regards,
Markus

Re: [v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]



On Sun, 17 Feb 2019, Markus Elfring wrote:

> >> If you would insist on the specification of such an assignment exclusion
> >> for a SmPL ellipsis:
> >> Can we agree on a correct order?
> >
> > I don't get your point.
>
> I propose to take another closer look at a bit of SmPL code.
>
>
> > There is no correct order.
>
> I have got an other software development view here.
>
>
> > Each order expresses something different.
>
> I agree to this information.
>
>
> > The order that is currently in the semantic patch is the one
> > that is more likely in practice.
>
> Please check once more.
>
> …
> +@search exists@
> +local idexpression id;
> +expression x,e,e1;
> +position p1,p2;
> …
> +@@
> +
> +id = of_find_device_by_node@p1(x)
> +... when != e = id
> …
>
> Or:
>
> …
> + ... when != id = e
> …
>
>
> Which SmPL specification will achieve the desired software behaviour?

The desired behavior is to check whether the allocated value is saved in
some other variable (typically a structure field) and thus it doesn't need
to be freed just because the original local variable goes out of scope at
the end of the function.  when != e = id achieves this behavior.

julia

Re: [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>> …
>> +@search exists@
>> +local idexpression id;
>> +expression x,e,e1;
>> +position p1,p2;
>> …
>> +@@
>> +
>> +id = of_find_device_by_node@p1(x)
>> +... when != e = id
>> …
>>
>> Or:
>>
>> …
>> + ... when != id = e
>> …
>>
>>
>> Which SmPL specification will achieve the desired software behaviour?
>
> The desired behavior is to check whether the allocated value is saved in
> some other variable (typically a structure field) and thus it doesn't need
> to be freed just because the original local variable goes out of scope at
> the end of the function.

I find this description reasonable to some degree.

(I am unsure if a programmer would like to fiddle with return value storage
in a data structure member from a local variable.)


> when != e = id achieves this behavior.

I can not agree to this view completely because of the meaning that is connected
with these variable identifiers.

Both metavariables share the kind “expression”. So I can imagine
that there is an intersection for the source code match possibility.
But one was intentionally restricted to the kind “local idexpression” so far.

Which data element should not get reassigned here (before a corresponding
null pointer check)?

Regards,
Markus