[Cocci] [PATCH v6] coccinelle: semantic code search for missing put_device()

[Cocci] [PATCH v6] coccinelle: semantic code search for missing put_device()

[Wen Yang]

The of_find_device_by_node() takes a reference to the underlying device
structure, we should release that reference.
The implementation of this semantic code search is:
In a function, for a local variable obtained by of_find_device_by_node(),
a, if it is released by a function such as
   put_device()/of_dev_put()/platform_device_put() after the last use,
   it is considered that there is no reference leak;
b, if it is passed back to the caller via
   dev_get_drvdata()/platform_get_drvdata()/get_device(), etc., the
   reference will be released in other functions, and the current function
   also considers that there is no reference leak;
c, for the rest of the situation, the current function should release the
   reference by calling put_device, this code search will report an error
   with a specific confidence.

By using this semantic code search, we have found some issues, such as:
commit 11907e9d3533 ("ASoC: fsl-asoc-card: fix object reference leaks in
fsl_asoc_card_probe")
commit a12085d13997 ("mtd: rawnand: atmel: fix possible object reference
leak")
commit 11493f26856a ("mtd: rawnand: jz4780: fix possible object reference
leak")

There are still dozens of reference leaks in the current kernel code.

Further, for the case of b, the object returned to other functions may also
have a reference leak, we will continue to develop other cocci scripts to
further check the reference leak.

Signed-off-by: Wen Yang <yellowriver2010@hotmail.com>
Reviewed-by: Julia Lawall <Julia.Lawall@lip6.fr>
Reviewed-by: Markus Elfring <Markus.Elfring@web.de>
Cc: Julia Lawall <Julia.Lawall@lip6.fr>
Cc: Gilles Muller <Gilles.Muller@lip6.fr>
Cc: Nicolas Palix <nicolas.palix@imag.fr>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Markus Elfring <Markus.Elfring@web.de>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Wen Yang <wen.yang99@zte.com.cn>
Cc: cheng.shengyu@zte.com.cn
Cc: cocci@systeme.lip6.fr
Cc: linux-kernel@vger.kernel.org
---
v6:
- to be double sure, replace &id->dev with (T)(&id->dev).
- long string literals can be accepted because of error message search concerns around a tool like grep
v5:
- exchange the word patch by code search.
- add a SPDX identifier.
- a split string literal can be unwanted.
- Change the content of the reported information.
v4:
- add Masahiro Yamada
- omit a blank line
- split the long message parameter
- reduce the number of metavariables
- Describe the implementation of the semantic patch,
  explain the scenarios it can detect,
  and further software development considerations. 
v3:
- reduction of a bit of redundant C code within SmPL search specifications.
- consider the message construction without using the extra Python variable msg.
v2:
- put exists after search, and then drop the when exists below.
- should not use the same e as in the when's below.
- Make a new type metavariable and use it to put a cast on the result of platform_get_drvdata.

 scripts/coccinelle/free/put_device.cocci | 55 ++++++++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100644 scripts/coccinelle/free/put_device.cocci

diff --git a/scripts/coccinelle/free/put_device.cocci b/scripts/coccinelle/free/put_device.cocci
new file mode 100644
index 000000000000..96e2508c0be1
--- /dev/null
+++ b/scripts/coccinelle/free/put_device.cocci
@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: GPL-2.0
+/// Find missing put_device for every of_find_device_by_node.
+///
+// Confidence: Moderate
+// Copyright: (C) 2018-2019 Wen Yang, ZTE.
+// Comments:
+// Options: --no-includes --include-headers
+
+virtual report
+virtual org
+
+@search exists@
+local idexpression id;
+expression x,e,e1;
+position p1,p2;
+type T,T1,T2;
+@@
+
+id = of_find_device_by_node@p1(x)
+... when != e = id
+if (id == NULL || ...) { ... return ...; }
+... when != put_device(&id->dev)
+    when != platform_device_put(id)
+    when != of_dev_put(id)
+    when != if (id) { ... put_device(&id->dev) ... }
+    when != e1 = (T)id
+    when != e1 = (T)(&id->dev)
+    when != e1 = get_device(&id->dev)
+    when != e1 = (T)platform_get_drvdata(id)
+(
+  return
+(    id
+|    (T1)dev_get_drvdata(&id->dev)
+|    (T2)platform_get_drvdata(id)
+);
+| return@p2 ...;
+)
+
+@script:python depends on report@
+p1 << search.p1;
+p2 << search.p2;
+@@
+
+coccilib.report.print_report(p2[0],
+			     "ERROR: missing put_device; call of_find_device_by_node on line "
+                             + p1[0].line
+                             + ", but without a corresponding object release within this function.")
+
+@script:python depends on org@
+p1 << search.p1;
+p2 << search.p2;
+@@
+
+cocci.print_main("of_find_device_by_node", p1)
+cocci.print_secs("needed put_device", p2)
-- 
2.20.1

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [PATCH v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

On Sat, 16 Feb 2019, Wen Yang wrote:

> The of_find_device_by_node() takes a reference to the underlying device
> structure, we should release that reference.
> The implementation of this semantic code search is:
> In a function, for a local variable obtained by of_find_device_by_node(),
> a, if it is released by a function such as
>    put_device()/of_dev_put()/platform_device_put() after the last use,
>    it is considered that there is no reference leak;
> b, if it is passed back to the caller via
>    dev_get_drvdata()/platform_get_drvdata()/get_device(), etc., the
>    reference will be released in other functions, and the current function
>    also considers that there is no reference leak;
> c, for the rest of the situation, the current function should release the
>    reference by calling put_device, this code search will report an error
>    with a specific confidence.
>
> By using this semantic code search, we have found some issues, such as:
> commit 11907e9d3533 ("ASoC: fsl-asoc-card: fix object reference leaks in
> fsl_asoc_card_probe")
> commit a12085d13997 ("mtd: rawnand: atmel: fix possible object reference
> leak")
> commit 11493f26856a ("mtd: rawnand: jz4780: fix possible object reference
> leak")
>
> There are still dozens of reference leaks in the current kernel code.
>
> Further, for the case of b, the object returned to other functions may also
> have a reference leak, we will continue to develop other cocci scripts to
> further check the reference leak.
>
> Signed-off-by: Wen Yang <yellowriver2010@hotmail.com>
> Reviewed-by: Julia Lawall <Julia.Lawall@lip6.fr>

Acked-by: Julia Lawall <julia.lawall@lip6.fr>

> Reviewed-by: Markus Elfring <Markus.Elfring@web.de>
> Cc: Julia Lawall <Julia.Lawall@lip6.fr>
> Cc: Gilles Muller <Gilles.Muller@lip6.fr>
> Cc: Nicolas Palix <nicolas.palix@imag.fr>
> Cc: Michal Marek <michal.lkml@markovi.net>
> Cc: Markus Elfring <Markus.Elfring@web.de>
> Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
> Cc: Wen Yang <wen.yang99@zte.com.cn>
> Cc: cheng.shengyu@zte.com.cn
> Cc: cocci@systeme.lip6.fr
> Cc: linux-kernel@vger.kernel.org
> ---
> v6:
> - to be double sure, replace &id->dev with (T)(&id->dev).
> - long string literals can be accepted because of error message search concerns around a tool like grep
> v5:
> - exchange the word patch by code search.
> - add a SPDX identifier.
> - a split string literal can be unwanted.
> - Change the content of the reported information.
> v4:
> - add Masahiro Yamada
> - omit a blank line
> - split the long message parameter
> - reduce the number of metavariables
> - Describe the implementation of the semantic patch,
>   explain the scenarios it can detect,
>   and further software development considerations.
> v3:
> - reduction of a bit of redundant C code within SmPL search specifications.
> - consider the message construction without using the extra Python variable msg.
> v2:
> - put exists after search, and then drop the when exists below.
> - should not use the same e as in the when's below.
> - Make a new type metavariable and use it to put a cast on the result of platform_get_drvdata.
>
>  scripts/coccinelle/free/put_device.cocci | 55 ++++++++++++++++++++++++
>  1 file changed, 55 insertions(+)
>  create mode 100644 scripts/coccinelle/free/put_device.cocci
>
> diff --git a/scripts/coccinelle/free/put_device.cocci b/scripts/coccinelle/free/put_device.cocci
> new file mode 100644
> index 000000000000..96e2508c0be1
> --- /dev/null
> +++ b/scripts/coccinelle/free/put_device.cocci
> @@ -0,0 +1,55 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/// Find missing put_device for every of_find_device_by_node.
> +///
> +// Confidence: Moderate
> +// Copyright: (C) 2018-2019 Wen Yang, ZTE.
> +// Comments:
> +// Options: --no-includes --include-headers
> +
> +virtual report
> +virtual org
> +
> +@search exists@
> +local idexpression id;
> +expression x,e,e1;
> +position p1,p2;
> +type T,T1,T2;
> +@@
> +
> +id = of_find_device_by_node@p1(x)
> +... when != e = id
> +if (id == NULL || ...) { ... return ...; }
> +... when != put_device(&id->dev)
> +    when != platform_device_put(id)
> +    when != of_dev_put(id)
> +    when != if (id) { ... put_device(&id->dev) ... }
> +    when != e1 = (T)id
> +    when != e1 = (T)(&id->dev)
> +    when != e1 = get_device(&id->dev)
> +    when != e1 = (T)platform_get_drvdata(id)
> +(
> +  return
> +(    id
> +|    (T1)dev_get_drvdata(&id->dev)
> +|    (T2)platform_get_drvdata(id)
> +);
> +| return@p2 ...;
> +)
> +
> +@script:python depends on report@
> +p1 << search.p1;
> +p2 << search.p2;
> +@@
> +
> +coccilib.report.print_report(p2[0],
> +			     "ERROR: missing put_device; call of_find_device_by_node on line "
> +                             + p1[0].line
> +                             + ", but without a corresponding object release within this function.")
> +
> +@script:python depends on org@
> +p1 << search.p1;
> +p2 << search.p2;
> +@@
> +
> +cocci.print_main("of_find_device_by_node", p1)
> +cocci.print_secs("needed put_device", p2)
> --
> 2.20.1
>
>
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

> In a function, for a local variable obtained by of_find_device_by_node(),

I got a software understanding where such a variable can not be obtained
from this function call.
The return value (like a pointer in this use case) can be stored there.


> v6:
> - to be double sure, replace &id->dev with (T)(&id->dev).

The support for data type casts is another interesting extension for
this source code analysis approach.
Further adjustments might become possible at other places of the presented SmPL script
after specific clarifications of previously mentioned implementation details.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

[Cocci] 答复: [v6] coccinelle: semantic code search for missing put_device()

[Wen Yang]

Hi Markus, 


> > In a function, for a local variable obtained by of_find_device_by_node(),
> 
> I got a software understanding where such a variable can not be obtained
> from this function call.
> The return value (like a pointer in this use case) can be stored there.
> 
> > v6:
> > - to be double sure, replace &id->dev with (T)(&id->dev).
> 
> The support for data type casts is another interesting extension for
> this source code analysis approach.
> Further adjustments might become possible at other places of the presented SmPL script
> after specific clarifications of previously mentioned implementation details.

First of all, thank you for your comments.

But please also refer to the examples of coccinelle, such as:
http://coccinelle.lip6.fr/rules/kmalloc.html
and 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/coccinelle/free/pci_free_consistent.cocci

You will find that there are differences between coccinelle and c.

 
Regards,
Wen
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] 答复: [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

> But please also refer to the examples of coccinelle, such as:
> http://coccinelle.lip6.fr/rules/kmalloc.html
> and
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/coccinelle/free/pci_free_consistent.cocci

These scripts for the semantic patch language show some software design possibilities.
They contain implementation details which can be also worth for additional
development considerations.
Will systematic refactoring become more interesting?


> You will find that there are differences between coccinelle and c.

Would you like to discuss any of them further?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [PATCH v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

> +@search exists@
> +local idexpression id;
> +expression x,e,e1;
> +position p1,p2;
> +type T,T1,T2;
> +@@
> +
> +id = of_find_device_by_node@p1(x)
> +... when != e = id

I suggest to increase your software development attention also for
another implementation detail.
Source code analysis triggers challenges for safe data flow handling.
the semantic patch language supports search specifications for
the exclusion of specific assignments.

Does this SmPL code contain a questionable order for the source
and target metavariables?
Can the following variant be more appropriate?

+ ... when != id = e


> +if (id == NULL || ...) { ... return ...; }
> +... when != put_device(&id->dev)
> +    when != platform_device_put(id)
> +    when != of_dev_put(id)
> +    when != if (id) { ... put_device(&id->dev) ... }
> +    when != e1 = (T)id

Would you like to avoid that the return value from the shown function call
gets overwritten in the variable before it was used once at least
(when a bit of extra C code is tolerated before a null pointer check)?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [PATCH v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

On Sun, 17 Feb 2019, Markus Elfring wrote:

> > +@search exists@
> > +local idexpression id;
> > +expression x,e,e1;
> > +position p1,p2;
> > +type T,T1,T2;
> > +@@
> > +
> > +id = of_find_device_by_node@p1(x)
> > +... when != e = id
>
> I suggest to increase your software development attention also for
> another implementation detail.
> Source code analysis triggers challenges for safe data flow handling.
> the semantic patch language supports search specifications for
> the exclusion of specific assignments.
>
> Does this SmPL code contain a questionable order for the source
> and target metavariables?
> Can the following variant be more appropriate?
>
> + ... when != id = e

This is possible, but I think unlikely.

>
>
> > +if (id == NULL || ...) { ... return ...; }
> > +... when != put_device(&id->dev)
> > +    when != platform_device_put(id)
> > +    when != of_dev_put(id)
> > +    when != if (id) { ... put_device(&id->dev) ... }
> > +    when != e1 = (T)id
>
> Would you like to avoid that the return value from the shown function call
> gets overwritten in the variable before it was used once at least
> (when a bit of extra C code is tolerated before a null pointer check)?

Indeed there should be a put then too, but again, it seems unlikely.

julia


>
> Regards,
> Markus
>
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [PATCH v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>>> +@search exists@
>>> +local idexpression id;
>>> +expression x,e,e1;
>>> +position p1,p2;
>>> +type T,T1,T2;
>>> +@@
>>> +
>>> +id = of_find_device_by_node@p1(x)
>>> +... when != e = id
>>
>> I suggest to increase your software development attention also for
>> another implementation detail.
>> Source code analysis triggers challenges for safe data flow handling.
>> the semantic patch language supports search specifications for
>> the exclusion of specific assignments.
>>
>> Does this SmPL code contain a questionable order for the source
>> and target metavariables?
>> Can the following variant be more appropriate?
>>
>> + ... when != id = e
>
> This is possible, but I think unlikely.

Would you dare to interpret my update suggestion (reordering of two identifiers)
as a required SmPL script correction?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [PATCH v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

On Sun, 17 Feb 2019, Markus Elfring wrote:

> >>> +@search exists@
> >>> +local idexpression id;
> >>> +expression x,e,e1;
> >>> +position p1,p2;
> >>> +type T,T1,T2;
> >>> +@@
> >>> +
> >>> +id = of_find_device_by_node@p1(x)
> >>> +... when != e = id
> >>
> >> I suggest to increase your software development attention also for
> >> another implementation detail.
> >> Source code analysis triggers challenges for safe data flow handling.
> >> the semantic patch language supports search specifications for
> >> the exclusion of specific assignments.
> >>
> >> Does this SmPL code contain a questionable order for the source
> >> and target metavariables?
> >> Can the following variant be more appropriate?
> >>
> >> + ... when != id = e
> >
> > This is possible, but I think unlikely.
>
> Would you dare to interpret my update suggestion (reordering of two identifiers)
> as a required SmPL script correction?

I didn't suggest to reorder anything.  Both are needed.

And, no I don't consider it to be a required suggestion.  In practice,
reassigning such a variable is very unlikely.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>> Would you dare to interpret my update suggestion (reordering of two identifiers)
>> as a required SmPL script correction?
>
> I didn't suggest to reorder anything.

This is obvious according to your acknowledgement for the sixth version
of this evolving SmPL script.


> Both are needed.

If you would insist on the specification of such an assignment exclusion
for a SmPL ellipsis:
Can we agree on a correct order?


> And, no I don't consider it to be a required suggestion.

Have we got a different view about an implementation detail at this place?


> In practice, reassigning such a variable is very unlikely.

This can be.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

On Sun, 17 Feb 2019, Markus Elfring wrote:

> >> Would you dare to interpret my update suggestion (reordering of two identifiers)
> >> as a required SmPL script correction?
> >
> > I didn't suggest to reorder anything.
>
> This is obvious according to your acknowledgement for the sixth version
> of this evolving SmPL script.
>
>
> > Both are needed.
>
> If you would insist on the specification of such an assignment exclusion
> for a SmPL ellipsis:
> Can we agree on a correct order?

I don't get your point.  There is no correct order.  Each order expresses
something different.  The order that is currently in the semantic patch is
the one that is more likely in practice.

julia

>
>
> > And, no I don't consider it to be a required suggestion.
>
> Have we got a different view about an implementation detail at this place?
>
>
> > In practice, reassigning such a variable is very unlikely.
>
> This can be.
>
> Regards,
> Markus
>
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>> If you would insist on the specification of such an assignment exclusion
>> for a SmPL ellipsis:
>> Can we agree on a correct order?
>
> I don't get your point.

I propose to take another closer look at a bit of SmPL code.


> There is no correct order.

I have got an other software development view here.


> Each order expresses something different.

I agree to this information.


> The order that is currently in the semantic patch is the one
> that is more likely in practice.

Please check once more.

…
+@search exists@
+local idexpression id;
+expression x,e,e1;
+position p1,p2;
…
+@@
+
+id = of_find_device_by_node@p1(x)
+... when != e = id
…

Or:

…
+ ... when != id = e
…


Which SmPL specification will achieve the desired software behaviour?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]



On Sun, 17 Feb 2019, Markus Elfring wrote:

> >> If you would insist on the specification of such an assignment exclusion
> >> for a SmPL ellipsis:
> >> Can we agree on a correct order?
> >
> > I don't get your point.
>
> I propose to take another closer look at a bit of SmPL code.
>
>
> > There is no correct order.
>
> I have got an other software development view here.
>
>
> > Each order expresses something different.
>
> I agree to this information.
>
>
> > The order that is currently in the semantic patch is the one
> > that is more likely in practice.
>
> Please check once more.
>
> …
> +@search exists@
> +local idexpression id;
> +expression x,e,e1;
> +position p1,p2;
> …
> +@@
> +
> +id = of_find_device_by_node@p1(x)
> +... when != e = id
> …
>
> Or:
>
> …
> + ... when != id = e
> …
>
>
> Which SmPL specification will achieve the desired software behaviour?

The desired behavior is to check whether the allocated value is saved in
some other variable (typically a structure field) and thus it doesn't need
to be freed just because the original local variable goes out of scope at
the end of the function.  when != e = id achieves this behavior.

julia

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>> …
>> +@search exists@
>> +local idexpression id;
>> +expression x,e,e1;
>> +position p1,p2;
>> …
>> +@@
>> +
>> +id = of_find_device_by_node@p1(x)
>> +... when != e = id
>> …
>>
>> Or:
>>
>> …
>> + ... when != id = e
>> …
>>
>>
>> Which SmPL specification will achieve the desired software behaviour?
>
> The desired behavior is to check whether the allocated value is saved in
> some other variable (typically a structure field) and thus it doesn't need
> to be freed just because the original local variable goes out of scope at
> the end of the function.

I find this description reasonable to some degree.

(I am unsure if a programmer would like to fiddle with return value storage
in a data structure member from a local variable.)


> when != e = id achieves this behavior.

I can not agree to this view completely because of the meaning that is connected
with these variable identifiers.

Both metavariables share the kind “expression”. So I can imagine
that there is an intersection for the source code match possibility.
But one was intentionally restricted to the kind “local idexpression” so far.

Which data element should not get reassigned here (before a corresponding
null pointer check)?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[wen.yang99]

[-- Attachment #1.1: Type: text/plain, Size: 1026 bytes --]

> > when != e = id achieves this behavior.
> 
> I can not agree to this view completely because of the meaning that is connected
> with these variable identifiers.
> 
> Both metavariables share the kind “expression”. So I can imagine
> that there is an intersection for the source code match possibility.
> But one was intentionally restricted to the kind “local idexpression” so far.
> 
> Which data element should not get reassigned here (before a corresponding
> null pointer check)?
> 

Thank you for your comments.
We did some experiments:
+id = of_find_device_by_node@p1(x)
+... when != e = id
...
Or:
...
+ ... when != id = e

The number of issuses found by these two methods is the same.
When != e = id achieves this behavior.

In addition, we feel that we should probably accept this patch first, use it to find more memory leaks, and solve the actual problems in the kernel code.
As for the patch itself, we can continue to pursue perfect in the process of using it to solve practical problems.

Regards,
Wen

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1295 bytes --]



On Mon, 18 Feb 2019, wen.yang99@zte.com.cn wrote:

> > > when != e = id achieves this behavior.
> >
> > I can not agree to this view completely because of the meaning that is connected
> > with these variable identifiers.
> >
> > Both metavariables share the kind “expression”. So I can imagine
> > that there is an intersection for the source code match possibility.
> > But one was intentionally restricted to the kind “local idexpression” so far.
> >
> > Which data element should not get reassigned here (before a corresponding
> > null pointer check)?
> >
>
> Thank you for your comments.
> We did some experiments:
> +id = of_find_device_by_node@p1(x)
> +... when != e = id
> ...
> Or:
> ...
> + ... when != id = e
>
> The number of issuses found by these two methods is the same.
> When != e = id achieves this behavior.

They are the same because neither issue arises.  I would have a hard time
saying which one is more reasonable to test, since both are extremely
unlikely.

julia


>
> In addition, we feel that we should probably accept this patch first, use it to find more memory leaks, and solve the actual problems in the kernel code.
> As for the patch itself, we can continue to pursue perfect in the process of using it to solve practical problems.
>
> Regards,
> Wen

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>>> Which data element should not get reassigned here (before a corresponding
>>> null pointer check)?
>>>
>>
>> Thank you for your comments.
>> We did some experiments:
>> +id = of_find_device_by_node@p1(x)
>> +... when != e = id
>> ...
>> Or:
>> ...
>> + ... when != id = e
>>
>> The number of issuses found by these two methods is the same.

This can be because these SmPL specifications share some source code search functionality.


>> When != e = id achieves this behavior.
>
> They are the same because neither issue arises.

You might not notice a difference from a specific source file selection so far.


> I would have a hard time saying which one is more reasonable to test,

I suggest to reconsider the interpretation of this software situation once more.


> since both are extremely unlikely.

I disagree to this view because two ellipses were intentionally specified
in published SmPL scripts.
So some software developers found these “special use cases” important enough.


>> In addition, we feel that we should probably accept this patch first,

I disagree to this imagination because I would prefer to integrate a source code variant
without a bug (which was copied from a version on 2013-05-08 by Petr Strnad).
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/coccinelle/free/pci_free_consistent.cocci?id=f7b167113753e95ae61383e234f8d10142782ace#n12

I hope that nicer run time behaviour can become also relevant here.


>> use it to find more memory leaks, and solve the actual problems in the kernel code.

Your are not hindered to achieve specific software improvements with evolving
development approaches while the clarification and the final integration
of useful scripts for the semantic patch language can take a bit longer.


>> As for the patch itself, we can continue to pursue perfect in the process
>> of using it to solve practical problems.

I am curious on how your attention will evolve further for the corresponding
software correctness.

1. How much will you care for the order of identifiers within the application
   of SmPL assignment exclusions?

2. Would you like to take additional data type restrictions into account?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[wen.yang99]

[-- Attachment #1.1: Type: text/plain, Size: 1298 bytes --]

> > I would have a hard time saying which one is more reasonable to test, 
> I suggest to reconsider the interpretation of this software situation once more.
> > since both are extremely unlikely.
> I disagree to this view because two ellipses were intentionally specified
> in published SmPL scripts.
> So some software developers found these “special use cases” important enough.
> >> In addition, we feel that we should probably accept this patch first,
> I disagree to this imagination because I would prefer to integrate a source code variant
> without a bug (which was copied from a version on 2013-05-08 by Petr Strnad).
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/coccinelle/free/pci_free_consistent.cocci?id=f7b167113753e95ae61383e234f8d10142782ace#n12
> I hope that nicer run time behaviour can become also relevant here.

Both cases are extremely unlikely.
Although we have tested these two methods in the existing kernel code,
considering the evolution of the kernel code, these special cases may occur, so we are willing to take them into account.
We plan to modify the code like this:

 id = of_find_device_by_node@p1(x)
-... when != e = id
+... when != e = (T)id
+    when != id = (T)e

Do you have any other questions?
Thanks.

Regards,
Wen

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1438 bytes --]



On Tue, 19 Feb 2019, wen.yang99@zte.com.cn wrote:

> > > I would have a hard time saying which one is more reasonable to test,
> > I suggest to reconsider the interpretation of this software situation once more.
> > > since both are extremely unlikely.
> > I disagree to this view because two ellipses were intentionally specified
> > in published SmPL scripts.
> > So some software developers found these “special use cases” important enough.
> > >> In addition, we feel that we should probably accept this patch first,
> > I disagree to this imagination because I would prefer to integrate a source code variant
> > without a bug (which was copied from a version on 2013-05-08 by Petr Strnad).
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/coccinelle/free/pci_free_consistent.cocci?id=f7b167113753e95ae61383e234f8d10142782ace#n12
> > I hope that nicer run time behaviour can become also relevant here.
>
> Both cases are extremely unlikely.
> Although we have tested these two methods in the existing kernel code,
> considering the evolution of the kernel code, these special cases may occur, so we are willing to take them into account.
> We plan to modify the code like this:
>
>  id = of_find_device_by_node@p1(x)
> -... when != e = id
> +... when != e = (T)id
> +    when != id = (T)e

This change is fine with me.

julia

>
> Do you have any other questions?
> Thanks.
>
> Regards,
> Wen

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>> Although we have tested these two methods in the existing kernel code,
>> considering the evolution of the kernel code, these special cases may occur, so we are willing to take them into account.
>> We plan to modify the code like this:
>>
>>  id = of_find_device_by_node@p1(x)
>> -... when != e = id
>> +... when != e = (T)id
>> +    when != id = (T)e
>
> This change is fine with me.

Thanks for another positive feedback on such software implementation details.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

> Do you have any other questions?

Obviously, yes.

I am curious if this development discussion and code review will trigger
further software adjustments.
I guess that you will need additional time to reconsider specific items
from recent feedback.

Will corrections become relevant for specifications in (assignment) exclusions
of the second SmPL ellipsis in the discussed script?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[wen.yang99]

[-- Attachment #1.1: Type: text/plain, Size: 1973 bytes --]

> > Do you have any other questions?
> 
> Obviously, yes.
> I am curious if this development discussion and code review will trigger
> further software adjustments.
> I guess that you will need additional time to reconsider specific items
> from recent feedback.
> 
> Will corrections become relevant for specifications in (assignment) exclusions
> of the second SmPL ellipsis in the discussed script?

Let's do some experiments with the code in the current kernel.
Let us take this code as an example:
Https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/sound/soc/fsl/imx-sgtl5000.c

1, Original code
static int imx_sgtl5000_probe(struct platform_device *pdev)
{
        struct device_node *np = pdev->dev.of_node;
        struct device_node *ssi_np, *codec_np;
        struct platform_device *ssi_pdev;
...
        ssi_pdev = of_find_device_by_node(ssi_np);
...
}

Our current patch will find the problem:
./sound/soc/fsl/imx-sgtl5000.c:169:1-7: ERROR: missing put_device; call of_find_device_by_node on line 105, but without a corresponding object release within this function.
./sound/soc/fsl/imx-sgtl5000.c:177:1-7: ERROR: missing put_device; call of_find_device_by_node on line 105, but without a corresponding object release within this function.

The problem can be detected by both of the following methods:
+id = of_find_device_by_node@p1(x)
+... when != e = id
...
Or:
...
+ ... when != id = e

2,  Suppose we change it to:
static int imx_sgtl5000_probe(struct platform_device *pdev)
{
        struct device_node *np = pdev->dev.of_node;
        struct device_node *ssi_np, *codec_np;
        struct platform_device *ssi_pdev;
...
        ssi_pdev = of_find_device_by_node(ssi_np);
...
        ssi_pdev = to_platform_device(XYZ);
}

But this time, only the first method can detect the problem.

The second method, although the false positive rate is lower, 
but the recall rate is reduced, we may miss some real issues.
Thanks,

Regards,
Wen

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>> Will corrections become relevant for specifications in (assignment) exclusions
>> of the second SmPL ellipsis in the discussed script?
>
> Let's do some experiments with the code in the current kernel.

It seems that you provided additional information for the adjustment
of when specifications according to the first SmPL ellipsis.

Other details are still in the waiting queue for the desired software clarification.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

> Do you have any other questions?

I would like to point another aspect out for further development considerations.

The initial assignment targets are (id)expressions in the discussed analysis
approach so far.
Would you like to care also for value (or pointer) initialisations by resource
allocation calls at the following source code places?

* Variable definition

* Designated initialiser


Will possible software extensions trigger interesting challenges?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] [v6] coccinelle: semantic code search for missing put_device()

[Markus Elfring]

>> Which data element should not get reassigned here (before a corresponding
>> null pointer check)?
>>
>
> Thank you for your comments.
> We did some experiments:
> +id = of_find_device_by_node@p1(x)
> +... when != e = id
> ...
> Or:
> ...
> + ... when != id = e
>
> The number of issuses found by these two methods is the same.

Would you like to clarify the circumstances a bit more under which
you would notice corresponding differences?


> When != e = id achieves this behavior.

I try another explanation approach for a potentially safer source code search.


1. If you would look at the following SmPL code lines again,
   I imagine that you can determine also an useful constraint for this place.

…
+ id = of_find_device_by_node@p1(x)
+ ... when != ?????? = ??????
+ if (!id || ...) { ... return ...; }
…


2. Petr Strnad expressed the need for another constraint in the commit
   “scripts: Coccinelle script for pci_free_consistent()” (from 2013-05-08).
   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/coccinelle/free/pci_free_consistent.cocci?id=f7b167113753e95ae61383e234f8d10142782ace#n12

   I find that there are additional software development challenges
   to consider around another desirable assignment exclusion from the place
   of the return value storage and the data processing possibilities
   behind the corresponding null pointer check.
   The previous approach worked with the metavariables types (or kinds)
   “expression” and “local idexpression id” to some degree so far.
   But it is expected that an undesirable missed reference release
   can be safely pointed out only if the received platform device pointer
   is used only within a found function implementation.

   I would interpret this data flow requirement in the way
   that the pointer should not be forwarded to a data structure
   with a scope from outside (global?) the found function implementation.
   How would you like to express such an aspect by the current (or future)
   means of the semantic patch language?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci