[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

Hello,

I came during my software development activities along a few Linux source files
where I got an advice by the script "checkpatch.pl" like the following.

WARNING: Prefer seq_puts to seq_printf

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/scripts/checkpatch.pl?id=29fbff8698fc0ac1a1d74584b258e0bf18b469f9#n5610


Joe Perches pointed out that the function "seq_putc" would be more appropriate
at a specific source code place.
https://patchwork.kernel.org/patch/9364455/
https://lkml.kernel.org/r/<1475771699.1914.10.camel@perches.com>


Now I am curious on how good the semantic patch language is applicable
for advanced source code transformations in similar use cases.
With which metavariables and corresponding programming interfaces can strings that
are passed to such functions be checked in more detail by the Coccinelle software?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Julia Lawall]

On Fri, 14 Oct 2016, SF Markus Elfring wrote:

> Hello,
>
> I came during my software development activities along a few Linux source files
> where I got an advice by the script "checkpatch.pl" like the following.
>
> WARNING: Prefer seq_puts to seq_printf
>
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/scripts/checkpatch.pl?id=29fbff8698fc0ac1a1d74584b258e0bf18b469f9#n5610
>
>
> Joe Perches pointed out that the function "seq_putc" would be more appropriate
> at a specific source code place.
> https://patchwork.kernel.org/patch/9364455/
> https://lkml.kernel.org/r/<1475771699.1914.10.camel@perches.com>
>
>
> Now I am curious on how good the semantic patch language is applicable
> for advanced source code transformations in similar use cases.
> With which metavariables and corresponding programming interfaces can strings that
> are passed to such functions be checked in more detail by the Coccinelle software?

The simplest would be to use python or ocaml code to make the checks that
are needed.  You can use cocci.include_match(False) in python or
Coccilib.include_match false in ocaml when your test detects that
something is not suitable.  coccinelle/demos/first.cocci may be helpful.

julia

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

>> With which metavariables and corresponding programming interfaces can strings that
>> are passed to such functions be checked in more detail by the Coccinelle software?
> 
> The simplest would be to use python or ocaml code to make the checks that
> are needed.  You can use cocci.include_match(False) in python or
> Coccilib.include_match false in ocaml when your test detects that
> something is not suitable.

Thanks for your suggestion.


> coccinelle/demos/first.cocci may be helpful.

I have got the impression that this small SmPL script demonstrates the usage
of the mentioned function from your software library only for a simple position variable.
I do not see the way so far how desired data could be extracted from a function
(or expression?) parameter. Would you like to make the introspection options a bit
clearer for the SmPL interface?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Julia Lawall]

> > coccinelle/demos/first.cocci may be helpful.
>
> I have got the impression that this small SmPL script demonstrates the usage
> of the mentioned function from your software library only for a simple position variable.
> I do not see the way so far how desired data could be extracted from a function
> (or expression?) parameter. Would you like to make the introspection options a bit
> clearer for the SmPL interface?

You can inherit and test properties of anything you like in python or
ocaml code.  If you inherit some kind of metavariable other than a
position variable, the term will be represented as a string.  But that
should be fine for detecting eg if a string has only one character.  If it
has multiple characters, you can do the include match false call, so that
in the last rule when yu inherit again the metavariable containing th
string, it won't be seen.

julia

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

> You can inherit and test properties of anything you like in python or
> ocaml code.  If you inherit some kind of metavariable other than a
> position variable, the term will be represented as a string.  But that
> should be fine for detecting eg if a string has only one character.

Your explanation sounds promising.


> If it has multiple characters, you can do the include match false call, so that
> in the last rule when yu inherit again the metavariable containing th
> string, it won't be seen.

I see another software development challenge then.

* I imagine that this "false call" will work for the exclusion of unwanted source
  code fragments.

* But I will also need a way to construct a source code change ("plus code" in
  the semantic patch) with OCaml or Python scripting, won't I?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Julia Lawall]

> * But I will also need a way to construct a source code change ("plus code" in
>   the semantic patch) with OCaml or Python scripting, won't I?

If you need to do this, you can follow the examples in
demos/camltococci.cocci and demos/pythontococci.cocci

julia

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

> But that should be fine for detecting eg if a string has only one character.

Did you come along similar change possibilities at other places already?

http://lxr.free-electrons.com/source/include/linux/seq_file.h?v=4.8#L114

??????????????????????????????????????????????????????
? update candidate ? 1. replacement ? 2. replacement ?
??????????????????????????????????????????????????????
? seq_printf       ? seq_puts       ? seq_putc       ?
??????????????????????????????????????????????????????


Would it make sense to consider any more source code search patterns here?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Julia Lawall]

On Fri, 14 Oct 2016, SF Markus Elfring wrote:

> > But that should be fine for detecting eg if a string has only one character.
>
> Did you come along similar change possibilities at other places already?
>
> http://lxr.free-electrons.com/source/include/linux/seq_file.h?v=4.8#L114
>
> ??????????????????????????????????????????????????????
> ? update candidate ? 1. replacement ? 2. replacement ?
> ??????????????????????????????????????????????????????
> ? seq_printf       ? seq_puts       ? seq_putc       ?
> ??????????????????????????????????????????????????????
>
>
> Would it make sense to consider any more source code search patterns here?

I'm not familiar with the details of this issue.

julia

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

>> Would it make sense to consider any more source code search patterns here?
> 
> I'm not familiar with the details of this issue.

Have you heard about that any other Linux developers dare to think about
further collateral software evolution around a function like "seq_printf"?

How interesting can it become to take also the POSIX variants into account?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Julia Lawall]

On Fri, 14 Oct 2016, SF Markus Elfring wrote:

> >> Would it make sense to consider any more source code search patterns here?
> >
> > I'm not familiar with the details of this issue.
>
> Have you heard about that any other Linux developers dare to think about
> further collateral software evolution around a function like "seq_printf"?
>
> How interesting can it become to take also the POSIX variants into account?

Same answer as above :)  I have no idea.

You can look at patches people have previously submitted and see to what
extent there is interest in this issue.

julia

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

> You can look at patches people have previously submitted and see to what
> extent there is interest in this issue.

I see another software development challenge here.

The discussed source code search pattern builds on a "general function"
like "seq_printf" as an update candidate from which each instance
(probably with a different prefix) is connected with a few replacement possibilities.
Did any other software developer fiddle with corresponding combinations
of function names?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Julia Lawall]

On Fri, 14 Oct 2016, SF Markus Elfring wrote:

> > You can look at patches people have previously submitted and see to what
> > extent there is interest in this issue.
>
> I see another software development challenge here.
>
> The discussed source code search pattern builds on a "general function"
> like "seq_printf" as an update candidate from which each instance
> (probably with a different prefix) is connected with a few replacement possibilities.
> Did any other software developer fiddle with corresponding combinations
> of function names?

No idea.  Look at the code and see what is available.

julia

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

>> Did any other software developer fiddle with corresponding combinations
>> of function names?
> 
> No idea.  Look at the code and see what is available.

Is it interesting to handle more than one combination efficiently?

Is a loop needed for corresponding record sets?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

> The simplest would be to use python or ocaml code to make the checks that
> are needed.  You can use cocci.include_match(False) in python or
> Coccilib.include_match false in ocaml when your test detects that
> something is not suitable.

Will it occasionally matter then to distinguish better if a specific source code
fragment or a complete source file should be excluded from an analysis run
on demand?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Julia Lawall]

On Sat, 15 Oct 2016, SF Markus Elfring wrote:

> > The simplest would be to use python or ocaml code to make the checks that
> > are needed.  You can use cocci.include_match(False) in python or
> > Coccilib.include_match false in ocaml when your test detects that
> > something is not suitable.
>
> Will it occasionally matter then to distinguish better if a specific source code
> fragment or a complete source file should be excluded from an analysis run
> on demand?

I have no idea what you mean by this.  But since I know very little about
this printing problem in general, I doubt that I can give an answer.  Try
it and see what requirements turn up.

julia

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

>> I do not see the way so far how desired data could be extracted from a function
>> (or expression?) parameter. Would you like to make the introspection options a bit
>> clearer for the SmPL interface?
> 
> You can inherit and test properties of anything you like in python or ocaml code.

I have got an SmPL approach working.


> If you inherit some kind of metavariable other than a position variable,
> the term will be represented as a string.

This software design aspect might look sufficient for a basic variant of the discussed
source code search pattern.


> But that should be fine for detecting eg if a string has only one character.

I have got a few concerns for further software development considerations.
How do you think about to support also an advanced interface by which it could be
determined if a preprocessor symbol would correspond to such a desired character
(instead of fiddling with regular expressions within a rule for the embedded programming
language script)?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Julia Lawall]

On Sat, 15 Oct 2016, SF Markus Elfring wrote:

> >> I do not see the way so far how desired data could be extracted from a function
> >> (or expression?) parameter. Would you like to make the introspection options a bit
> >> clearer for the SmPL interface?
> >
> > You can inherit and test properties of anything you like in python or ocaml code.
>
> I have got an SmPL approach working.
>
>
> > If you inherit some kind of metavariable other than a position variable,
> > the term will be represented as a string.
>
> This software design aspect might look sufficient for a basic variant of the discussed
> source code search pattern.
>
>
> > But that should be fine for detecting eg if a string has only one character.
>
> I have got a few concerns for further software development considerations.
> How do you think about to support also an advanced interface by which it could be
> determined if a preprocessor symbol would correspond to such a desired character
> (instead of fiddling with regular expressions within a rule for the embedded programming
> language script)?

What C code are you concerned about exactly?

julia

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

>> Will it occasionally matter then to distinguish better if a specific source code
>> fragment or a complete source file should be excluded from an analysis run
>> on demand?
> 
> I have no idea what you mean by this.

I assume that there are more chances.

I am just unsure around the application of a function call like
"cocci.include_match(False)" so far.


> But since I know very little about this printing problem in general,

It is not a "problem". - Is it just the beginning for another software development
challenge (or adventure)?


> I doubt that I can give an answer.

I imagine that we can exchange some ideas to improve the affected software
situation further, can't we?


> Try it and see what requirements turn up.

Which items will be picked up for further clarification?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

>> I have got a few concerns for further software development considerations.
>> How do you think about to support also an advanced interface by which it could be
>> determined if a preprocessor symbol would correspond to such a desired character
>> (instead of fiddling with regular expressions within a rule for the embedded programming
>> language script)?
> 
> What C code are you concerned about exactly?

I guess that this "story" affects some general aspects. After I got a better understanding
of related software dependencies, I begin to think also about further possibilities to
make such a source code search pattern safer and more efficient.
How much would you like to increase the support for data processing around macros
with your software?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Julia Lawall]

On Sat, 15 Oct 2016, SF Markus Elfring wrote:

> >> I have got a few concerns for further software development considerations.
> >> How do you think about to support also an advanced interface by which it could be
> >> determined if a preprocessor symbol would correspond to such a desired character
> >> (instead of fiddling with regular expressions within a rule for the embedded programming
> >> language script)?
> >
> > What C code are you concerned about exactly?
>
> I guess that this "story" affects some general aspects. After I got a better understanding
> of related software dependencies, I begin to think also about further possibilities to
> make such a source code search pattern safer and more efficient.
> How much would you like to increase the support for data processing around macros
> with your software?

Ther are no particular plans in this direction.

julia

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

>> How much would you like to increase the support for data processing around macros
>> with your software?
> 
> Ther are no particular plans in this direction.

How does this kind of feedback fit to an information like ?Add identifier list metavariables
for #define parameter lists? from the recent software release?

Regards,
Markus

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[Julia Lawall]

On Sat, 15 Oct 2016, SF Markus Elfring wrote:

> >> How much would you like to increase the support for data processing around macros
> >> with your software?
> >
> > Ther are no particular plans in this direction.
>
> How does this kind of feedback fit to an information like ?Add identifier list metavariables
> for #define parameter lists? from the recent software release?

That was done already, so it's not a plan.  Furthermore, it was not
planned for, the need arose.  Need, not hypothetical need.

julia

[Cocci] Replacing calls of general functions by more specific ones with SmPL

[SF Markus Elfring]

>> Have you heard about that any other Linux developers dare to think about
>> further collateral software evolution around a function like "seq_printf"?
>>
>> How interesting can it become to take also the POSIX variants into account?
> 
> Same answer as above :)  I have no idea.

Do you eventually know any source code search databases which could be queried
for function names with the suffix "_printf" (or similar wordings)?

Regards,
Markus