[cocci] List global variables.

[cocci] List global variables.

[Alessandro Carminati]

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]

Hello,
I need to build a tool that just lists the global variables for a given c
project.
It appeared to me to be a straight simple operation using coccinelle, still
the script I wrote does not return the expected values.
here the script
```
@g@
type T;
global idexpression T x;
@@
x

@script:python@
x << g.x;
@@
print (x)
```
And this is the simple file I used to test it.
```
#include <stdio.h>
#include <stdlib.h>

static char glid[3];
int cnt;

int function(int a){
return a;
}

int main(){
char *p;
int i,c;
p=malloc(100);
i=function(9);
c=sprintf(p, "Nice text%d\n", i);
puts(p);
free(p);
return 0;
}
```
my expectation was to get "glid" and "cnt" as product for the computation,
instead I just get "function".

Could anybody clarify why this?

cheers
Alessandro

[-- Attachment #2: Type: text/html, Size: 1329 bytes --]

Re: [cocci] List global variables.

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]



On Mon, 9 May 2022, Alessandro Carminati wrote:

> Hello,I need to build a tool that just lists the global variables for a
> given c project.
> It appeared to me to be a straight simple operation using coccinelle, still
> the script I wrote does not return the expected values.
> here the script
> ```
> @g@
> type T;
> global idexpression T x;
> @@
> x
>
> @script:python@
> x << g.x;
> @@
> print (x)
> ```
> And this is the simple file I used to test it.
> ```
> #include <stdio.h>
> #include <stdlib.h>
>
> static char glid[3];
> int cnt;
>
> int function(int a){
> return a;
> }
>
> int main(){
> char *p;
> int i,c;
> p=malloc(100);
> i=function(9);
> c=sprintf(p, "Nice text%d\n", i);
> puts(p);
> free(p);
> return 0;
> }
> ```
> my expectation was to get "glid" and "cnt" as product for the computation,
> instead I just get "function".
>  
> Could anybody clarify why this?

A "global" variable is a variable that is referenced by a function but is
not declared by that function.

Actually, glid and cnt are not expressions.  They are only identifiers.
So they are not matched by idexpression.

You can do what you want using some script code.  I will send you a
solution in a few minutes (I actually wanted to do the same thing :).

julia

Re: [cocci] List global variables.

[Julia Lawall]

On Mon, 9 May 2022, Alessandro Carminati wrote:

> Hello,I need to build a tool that just lists the global variables for a
> given c project.

@r@
type T;
identifier i;
position p : script:python(i) { p[0].current_element == i};
@@

*T i@p;

The * is just for illustration.  You can do whatever you want with i.

julia

[cocci] Fwd: List global variables.

[Alessandro Carminati]

[-- Attachment #1: Type: text/plain, Size: 975 bytes --]

Hello Julia,

The solution you sent indeed does what I expected.
Although I needed a solution for this specific case, and yours fit my needs
perfectly, I'd love it if you wanted to explain it to me.
Would you mind add a short explanation of this statement:
`position p : script:python(i) { p[0].current_element == i};`
I probably have a simplified understanding of what a position is. Maybe it
is worth digging deeper and having a more concrete knowledge. Any read you
want to suggest to me?

Thank you for your time.
Alessandro




Il giorno lun 9 mag 2022 alle ore 12:17 Julia Lawall <julia.lawall@inria.fr>
ha scritto:

>
>
> On Mon, 9 May 2022, Alessandro Carminati wrote:
>
> > Hello,I need to build a tool that just lists the global variables for a
> > given c project.
>
> @r@
> type T;
> identifier i;
> position p : script:python(i) { p[0].current_element == i};
> @@
>
> *T i@p;
>
> The * is just for illustration.  You can do whatever you want with i.
>
> julia
>

[-- Attachment #2: Type: text/html, Size: 1667 bytes --]

Re: [cocci] Fwd: List global variables.

[Julia Lawall]

On Mon, 9 May 2022, Alessandro Carminati wrote:

> Hello Julia,
> The solution you sent indeed does what I expected.
> Although I needed a solution for this specific case, and yours fit my needs
> perfectly, I'd love it if you wanted to explain it to me.
> Would you mind add a short explanation of this statement:
> `position p : script:python(i) { p[0].current_element == i};`
> I probably have a simplified understanding of what a position is. Maybe it
> is worth digging deeper and having a more concrete knowledge. Any read you
> want to suggest to me?

Position variable have a collection of information about the position,
including the name of the containing object.  So if the name of the
containing object is the same as the name of the declared variable, then I
consider that the variable is a global variable.  If the name of the
containing object is different, then the declaration is inside a function
definition.

I think that if you have done make install for Coccinelle, then you should
be able to do man Coccilib and then find out about the fields for position
variable.  You may also be able to use reflection in python to get a list
of the field names.

julia

Re: [cocci] List global variables with SmPL

[Markus Elfring]

[-- Attachment #1: Type: text/plain, Size: 1116 bytes --]


> The solution you sent indeed does what I expected.


I doubt that it fits to your initial source code analysis desire.

Are your expectations still evolving (also for your data processing needs)?



> Would you mind add a short explanation of this statement:
> `position p : script:python(i) { p[0].current_element == i};`
> I probably have a simplified understanding of what a position is.

I imagine that known information sources can help further.

https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5069eaeadd731ecdd99e7a6f4465c286a2792354/docs/manual/cocci_syntax.tex#L410
https://github.com/coccinelle/coccinelle/blob/ae337fce1512ff15aabc3ad5b6d2e537f97ab62a/docs/manual/cocci_syntax.tex#L410



> Maybe it is worth digging deeper and having a more concrete knowledge.


Probably, yes.



> Any read you want to suggest to me?


Related links for example:

https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5069eaeadd731ecdd99e7a6f4465c286a2792354/docs/Coccilib.3cocci#L5
https://github.com/coccinelle/coccinelle/blob/57cbff0c5768e22bb2d8c20e8dae74294515c6b3/docs/Coccilib.3cocci#L5



Regards,
Markus

[-- Attachment #2: Type: text/html, Size: 3439 bytes --]

Re: [cocci] List global variables with SmPL

[Alessandro Carminati]

[-- Attachment #1: Type: text/plain, Size: 2169 bytes --]

Hello Markus,


Il giorno lun 9 mag 2022 alle ore 21:23 Markus Elfring <
Markus.Elfring@web.de> ha scritto:

>
> The solution you sent indeed does what I expected.
>
>
> I doubt that it fits to your initial source code analysis desire.
>
> Are your expectations still evolving (also for your data processing needs)?
>
Indeed Julia's proposed code does not solve all the situations I encounter
now  in the kernel source analysis.
It reports not only global variable names but also function names.
It confuses sparse annotation and a few other macros with variable names.
I also have trouble with "extern" declared variables that appear multiple
times.
Still, it is far better than I was up to, And it allows me to improve the
solution by adding statements and rules to make it better suit my use case.
Currently, I'm using this, but I still have duplicates due to "extern"
declarations.
```
@func@
type T;
identifier i;
position p : script:python(i) { p[0].current_element == i};
@@

T i(...)@p;


@r depends on !func@
type T;
identifier i;
expression E;
position p : script:python(i) { p[0].current_element == i};
@@

(
 T i@p;
|
 T i@p=E;
)

@script:python@
i << r.i;
@@
print (i)
```

>
>
> Would you mind add a short explanation of this statement:
> `position p : script:python(i) { p[0].current_element == i};`
> I probably have a simplified understanding of what a position is.
>
> I imagine that known information sources can help further.
>
>
> https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5069eaeadd731ecdd99e7a6f4465c286a2792354/docs/manual/cocci_syntax.tex#L410
>
> https://github.com/coccinelle/coccinelle/blob/ae337fce1512ff15aabc3ad5b6d2e537f97ab62a/docs/manual/cocci_syntax.tex#L410
>
Thanks for this.

>
>
>
> Maybe it is worth digging deeper and having a more concrete knowledge.
>
>
> Probably, yes.
>
>
>
> Any read you want to suggest to me?
>
>
> Related links for example:
>
>
> https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5069eaeadd731ecdd99e7a6f4465c286a2792354/docs/Coccilib.3cocci#L5
>
> https://github.com/coccinelle/coccinelle/blob/57cbff0c5768e22bb2d8c20e8dae74294515c6b3/docs/Coccilib.3cocci#L5
>
>
>
> Regards,
> Markus
>

[-- Attachment #2: Type: text/html, Size: 5096 bytes --]

Re: [cocci] List global variables with SmPL

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 3630 bytes --]



On Tue, 10 May 2022, Alessandro Carminati wrote:

> Hello Markus,
>
>
> Il giorno lun 9 mag 2022 alle ore 21:23 Markus Elfring
> <Markus.Elfring@web.de> ha scritto:
>
>             The solution you sent indeed does what I expected.
>
>
> I doubt that it fits to your initial source code analysis desire.
>
> Are your expectations still evolving (also for your data processing
> needs)?
>
> Indeed Julia's proposed code does not solve all the situations I encounter
> now  in the kernel source analysis.
> It reports not only global variable names but also function names.
> It confuses sparse annotation and a few other macros with variable names.
> I also have trouble with "extern" declared variables that appear multiple
> times.
> Still, it is far better than I was up to, And it allows me to improve the
> solution by adding statements and rules to make it better suit my use case.
> Currently, I'm using this, but I still have duplicates due to "extern"
> declarations.
> ```
> @func@
> type T;
> identifier i;
> position p : script:python(i) { p[0].current_element == i};
> @@
>
> T i(...)@p;
>
>
> @r depends on !func@
> type T;
> identifier i;
> expression E;
> position p : script:python(i) { p[0].current_element == i};
> @@

Thanks for the feedback.  The strategy of !func is a good idea, but it is
not correct.  There is no particular relation between the func rule and
this one.  Thus as soon as a file contains code that matches the func
rule, the file will be ignored.

One solution would be to check that the position of i is not the same as
the position of a name matched by the func rule.  For that, you could move
the @p in the func rule from the ) to the function name, and then in this
rule put as the first metavariable line:

position q != func.p;

and then in the matches below put i@q@p (ensure that is at a position
that is both different than any func.p and that it satisfies the
current_element constraint).

But in this case, there is a solution that is even simpler:

(
T i(...);
|
T i@p;
|
T i@p=E;
)

Indeed, I think it should be sufficient to say:

(
T i(...);
|
T i@p=...;
)

You can run spatch --parse-cocci to see what happens with the rewriting
due to isomorphisms to see if the simpler rule is sufficient.

Or maybe you want:

(
T i(...);
|
extern T i;
|
T i@p=...;
)


>
> (
>  T i@p;
> |
>  T i@p=E;
> )
>
> @script:python@
> i << r.i;
> @@
> print (i)

Maybe you also want to inherit the position variable and print the file
and line number of the declaration?

julia

> ```
>
>
>
>             Would you mind add a short explanation of this
>             statement:
>             `position p : script:python(i) {
>             p[0].current_element == i};`
>             I probably have a simplified understanding of what a
>             position is.
>
> I imagine that known information sources can help further.
>
> https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5069eaeadd731ecdd99e7a
> 6f4465c286a2792354/docs/manual/cocci_syntax.tex#L410
> https://github.com/coccinelle/coccinelle/blob/ae337fce1512ff15aabc3ad5b6d2e
> 537f97ab62a/docs/manual/cocci_syntax.tex#L410
>
> Thanks for this.  
>
>
>
>
>             Maybe it is worth digging deeper and having a more
>             concrete knowledge.
>
>
> Probably, yes.
>
>
>
>       Any read you want to suggest to me?
>
>
> Related links for example:
>
> https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5069eaeadd731ecdd99e7a
> 6f4465c286a2792354/docs/Coccilib.3cocci#L5
> https://github.com/coccinelle/coccinelle/blob/57cbff0c5768e22bb2d8c20e8dae7
> 4294515c6b3/docs/Coccilib.3cocci#L5
>
>
>
> Regards,
> Markus
>
>
>

Re: [cocci] List global variables with SmPL

[Alessandro Carminati]

[-- Attachment #1: Type: text/plain, Size: 5272 bytes --]

Hello Julia,
I modified the coccinelle script in accordance with your suggestions (at
least, my understanding of those).
The script I'm currently using is like this:
```
@excluded@
type T;
identifier i;
position p;
@@

(
 T i@p(...);
|
 extern T i@p;
)


@r@
type T;
identifier i;
expression E;
position q != excluded.p;
position p : script:python(i) { p[0].current_element == i};
@@

(
 T i@q@p;
|
 T i@q@p=E;
)

@script:python@
i << r.i;
@@
print (i)
```
The rule "excluded" should find what I need to be excluded.
Still, the script has trouble with statements like this:
```
__printf(3, 4) __cold
void _dev_printk(const char *level, const struct device *dev,
                 const char *fmt, ...);
```
And it reports "_dev_printk" as a global variable.
There are also problems with some "__randomize_layout" statements, but I've
not yet isolated the minimal statement that can report "__randomize_layout"
as the global variable name.
My assumption is that I can add defines as done in the init_defs_builtins:
/usr/local/lib/coccinelle/standard.h, but using the -define argument with a
file containing these defines does not solve the problem.
Sorry for keeping this thread long, but your advice would be very
appreciated.

cheers
Alessandro



Il giorno mar 10 mag 2022 alle ore 10:06 Julia Lawall <julia.lawall@inria.fr>
ha scritto:

>
>
> On Tue, 10 May 2022, Alessandro Carminati wrote:
>
> > Hello Markus,
> >
> >
> > Il giorno lun 9 mag 2022 alle ore 21:23 Markus Elfring
> > <Markus.Elfring@web.de> ha scritto:
> >
> >             The solution you sent indeed does what I expected.
> >
> >
> > I doubt that it fits to your initial source code analysis desire.
> >
> > Are your expectations still evolving (also for your data processing
> > needs)?
> >
> > Indeed Julia's proposed code does not solve all the situations I
> encounter
> > now  in the kernel source analysis.
> > It reports not only global variable names but also function names.
> > It confuses sparse annotation and a few other macros with variable names.
> > I also have trouble with "extern" declared variables that appear multiple
> > times.
> > Still, it is far better than I was up to, And it allows me to improve the
> > solution by adding statements and rules to make it better suit my use
> case.
> > Currently, I'm using this, but I still have duplicates due to "extern"
> > declarations.
> > ```
> > @func@
> > type T;
> > identifier i;
> > position p : script:python(i) { p[0].current_element == i};
> > @@
> >
> > T i(...)@p;
> >
> >
> > @r depends on !func@
> > type T;
> > identifier i;
> > expression E;
> > position p : script:python(i) { p[0].current_element == i};
> > @@
>
> Thanks for the feedback.  The strategy of !func is a good idea, but it is
> not correct.  There is no particular relation between the func rule and
> this one.  Thus as soon as a file contains code that matches the func
> rule, the file will be ignored.
>
> One solution would be to check that the position of i is not the same as
> the position of a name matched by the func rule.  For that, you could move
> the @p in the func rule from the ) to the function name, and then in this
> rule put as the first metavariable line:
>
> position q != func.p;
>
> and then in the matches below put i@q@p (ensure that is at a position
> that is both different than any func.p and that it satisfies the
> current_element constraint).
>
> But in this case, there is a solution that is even simpler:
>
> (
> T i(...);
> |
> T i@p;
> |
> T i@p=E;
> )
>
> Indeed, I think it should be sufficient to say:
>
> (
> T i(...);
> |
> T i@p=...;
> )
>
> You can run spatch --parse-cocci to see what happens with the rewriting
> due to isomorphisms to see if the simpler rule is sufficient.
>
> Or maybe you want:
>
> (
> T i(...);
> |
> extern T i;
> |
> T i@p=...;
> )
>
>
> >
> > (
> >  T i@p;
> > |
> >  T i@p=E;
> > )
> >
> > @script:python@
> > i << r.i;
> > @@
> > print (i)
>
> Maybe you also want to inherit the position variable and print the file
> and line number of the declaration?
>
> julia
>
> > ```
> >
> >
> >
> >             Would you mind add a short explanation of this
> >             statement:
> >             `position p : script:python(i) {
> >             p[0].current_element == i};`
> >             I probably have a simplified understanding of what a
> >             position is.
> >
> > I imagine that known information sources can help further.
> >
> >
> https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5069eaeadd731ecdd99e7a
> > 6f4465c286a2792354/docs/manual/cocci_syntax.tex#L410
> >
> https://github.com/coccinelle/coccinelle/blob/ae337fce1512ff15aabc3ad5b6d2e
> > 537f97ab62a/docs/manual/cocci_syntax.tex#L410
> >
> > Thanks for this.
> >
> >
> >
> >
> >             Maybe it is worth digging deeper and having a more
> >             concrete knowledge.
> >
> >
> > Probably, yes.
> >
> >
> >
> >       Any read you want to suggest to me?
> >
> >
> > Related links for example:
> >
> >
> https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5069eaeadd731ecdd99e7a
> > 6f4465c286a2792354/docs/Coccilib.3cocci#L5
> >
> https://github.com/coccinelle/coccinelle/blob/57cbff0c5768e22bb2d8c20e8dae7
> > 4294515c6b3/docs/Coccilib.3cocci#L5
> >
> >
> >
> > Regards,
> > Markus
> >
> >
> >

[-- Attachment #2: Type: text/html, Size: 7593 bytes --]

Re: [cocci] List global variables with SmPL

[Julia Lawall]

> There are also problems with some "__randomize_layout" statements, but I've not yet isolated the minimal statement that can report "__randomize_layout" as the global variable name.
> My assumption is that I can add defines as done in the init_defs_builtins: /usr/local/lib/coccinelle/standard.h, but using the -define argument with a file containing these defines does not solve the
> problem.
> Sorry for keeping this thread long, but your advice would be very appreciated.

-define is to indicate that a configuration variable is true.  You want
--macro-file <file>

julia

Re: [cocci] List global variables with SmPL

[Alessandro Carminati]

[-- Attachment #1: Type: text/plain, Size: 1695 bytes --]

Hello,

Il giorno mar 10 mag 2022 alle ore 11:24 Julia Lawall <julia.lawall@inria.fr>
ha scritto:

> > There are also problems with some "__randomize_layout" statements, but
> I've not yet isolated the minimal statement that can report
> "__randomize_layout" as the global variable name.
> > My assumption is that I can add defines as done in the
> init_defs_builtins: /usr/local/lib/coccinelle/standard.h, but using the
> -define argument with a file containing these defines does not solve the
> > problem.
> > Sorry for keeping this thread long, but your advice would be very
> appreciated.
>
> -define is to indicate that a configuration variable is true.  You want
> --macro-file <file>
>
--macro-file is probably what I needed. But for some reason, it did not do
what I expected. Appending "#define __randomize_layout" to the
/usr/local/lib/coccinelle/standard.h does what I expect and the
"__randomize_layout" is removed from the results, but the argument
 "--macro-file local_def.h" with the file containing the same single
statement does not.

Also, for the other problem I had with the statement:
```
__printf(3, 4) __cold
void _dev_printk(const char *level, const struct device *dev,
         const char *fmt, ...);
```
I assumed the problem to lay in the first two defines "__printf(3, 4)" and
"__cold", but at least the "printf(a,b)" exists already in the default
standard.h, and adding the second does not make disappear the _dev_printk
from the results.
Instead, it seems my problem pertains to the variadic functions. Removing
the three dots by the function declaration, the entry disappears from the
results.
I have no clue here. Again, help is needed.
Thank you



>
> julia
>

[-- Attachment #2: Type: text/html, Size: 2582 bytes --]

Re: [cocci] List global variables with SmPL

[Julia Lawall]

Here is a suggested solution for all of the problems reported so far:

@r@
type T;
identifier i;
expression E;
position p : script:python(i) { p[0].current_element == i };
attribute name __randomize_layout;
@@

(
T i(...);
|
T i(...,......);
|
extern T i;
|
T i@p;
|
T i@p=E;
)

@script:python@
i << r.i;
p << r.p;
@@
print (i)

----------------------

The problem with __randomize_layout is that the Coccinelle parser is not
recognizing it as an attribute.  You can force that by putting a
declaration in the semantic patch.

It seems that ... in a function parameter list does not match the ... in C
for a variable list of arguments.  That ... can be matched explicitly by
...... so I have added another case with that.

I als combined all of the patterns into one rule.  By inheriting the
position varaible p into the python rule at the end, that python rule is
only executed if p is defined, which makes it select the cases of interest
and ignore the others.

julia

Re: [cocci] List global variables with SmPL

[Markus Elfring]

> Here is a suggested solution for all of the problems reported so far:
>
> @r@
> type T;
> identifier i;
> expression E;
> position p : script:python(i) { p[0].current_element == i };
> attribute name __randomize_layout;
> @@
>
> (
> T i(...);
> |
> T i(...,......);
> |
> extern T i;
> |
> T i@p;
> |
> T i@p=E;
> )


I find that this SmPL code needs further clarifications.


The information “… current_element is the name of the function containing the matched position; …”
is provided by the manual for the Coccilib module.

* How can a function name be referenced if data should be determined for
  the global scope?

* Why is a comparison attempted in the scripted constraint at all
  if the passed identifier would refer to a variable name according to the metavariable “i”?


> @script:python@
> i << r.i;
> p << r.p;
> @@
> print (i)
…
> The problem with __randomize_layout is that the Coccinelle parser is not
> recognizing it as an attribute.  You can force that by putting a
> declaration in the semantic patch.
>
> It seems that ... in a function parameter list does not match the ... in C
> for a variable list of arguments.  That ... can be matched explicitly by
> ...... so I have added another case with that.
>
> I als combined all of the patterns into one rule.  By inheriting the
> position varaible p into the python rule at the end, that python rule is
> only executed if p is defined, which makes it select the cases of interest
> and ignore the others.


Will these details trigger any improvements for the software documentation?

Regards,
Markus

Re: [cocci] List global variables with SmPL

[Markus Elfring]

> The information “… current_element is the name of the function containing the matched position; …”
> is provided by the manual for the Coccilib module.
>
> * How can a function name be referenced if data should be determined for
>   the global scope?
>
> * Why is a comparison attempted in the scripted constraint at all
>   if the passed identifier would refer to a variable name according to the metavariable “i”?


Would you get into the mood to clarify affected technical details any more?

Will the handling of source code within the global scope get additional development attention
together with further evolution of the semantic patch language (Coccinelle software)?


Regards,
Markus

Re: [cocci] List global variables with SmPL

[Alessandro Carminati]

[-- Attachment #1: Type: text/plain, Size: 3032 bytes --]

I'm planning to stop bugging this mailing list, but before that, please
allow me a final question. That is related to what I have done so far.
My current implementation is the following.
```
@excluded@
type T;
identifier i;
position p;
@@

(
 T i@p(...);
|
 extern T i@p;
|
 T i@p(...,......);
)


@r@
type T;
identifier i;
expression E;
position q != excluded.p;
position p : script:python(i) { p[0].current_element == i};
attribute name __randomize_layout;
@@

(
 T i@q@p;
|
 T i@q@p=E;
)

@rr@
type T;
identifier i;
expression E;
attribute name __randomize_layout;
@@

(
 T i;
|
 T i=E;
)

@script:python@
i << excluded.i;
@@
print (i)
```
It is a suboptimal version of Julia's last proposition.
It happens that in a file a statement I expect to be matched is not
detected.
Troubleshooting the issue, I see a behavior that does not fit the model I
have in my mind. This suggests to me that things (coccinelle under the hood
mechanisms) are more complicated than I think.
back to the point:
Running the script and printing only the rule "rr", which is the same as
"r" without positions, the results presents a set of entries containing the
entry I'm interested in.
Running the same script but printing only the rule "excluded", I see that
the result is an empty set.
Finally, running the script using only the "r" rule, the entry I'm
interested in is not there.

Surprisingly (for me), removing the position constraints coming from the
rule "excluded" (position q != excluded.p;) from the rule "r", the entry
appears.
My question then is the following: How can the rule "excluded" that
presents an empty set influence the rule "r"?
Speculating Julia's words on a previous answer I got, I guessed that
somehow an empty set could be treated differently, so I added a line into
the target c file to make at least "excluded" match one line. But it didn't
make any difference.

Alessandro



Il giorno mar 10 mag 2022 alle ore 12:39 Julia Lawall <julia.lawall@inria.fr>
ha scritto:

> Here is a suggested solution for all of the problems reported so far:
>
> @r@
> type T;
> identifier i;
> expression E;
> position p : script:python(i) { p[0].current_element == i };
> attribute name __randomize_layout;
> @@
>
> (
> T i(...);
> |
> T i(...,......);
> |
> extern T i;
> |
> T i@p;
> |
> T i@p=E;
> )
>
> @script:python@
> i << r.i;
> p << r.p;
> @@
> print (i)
>
> ----------------------
>
> The problem with __randomize_layout is that the Coccinelle parser is not
> recognizing it as an attribute.  You can force that by putting a
> declaration in the semantic patch.
>
> It seems that ... in a function parameter list does not match the ... in C
> for a variable list of arguments.  That ... can be matched explicitly by
> ...... so I have added another case with that.
>
> I als combined all of the patterns into one rule.  By inheriting the
> position varaible p into the python rule at the end, that python rule is
> only executed if p is defined, which makes it select the cases of interest
> and ignore the others.
>
> julia
>

[-- Attachment #2: Type: text/html, Size: 4017 bytes --]

Re: [cocci] List global variables with SmPL

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 3603 bytes --]



On Wed, 11 May 2022, Alessandro Carminati wrote:

> I'm planning to stop bugging this mailing list, but before that, please allow me a final question. That is related to what I have done so far.
> My current implementation is the following.
> ```
> @excluded@
> type T;
> identifier i;
> position p;
> @@
>
> (
>  T i@p(...);
> |
>  extern T i@p;
> |
>  T i@p(...,......);
> )
>
>
> @r@
> type T;
> identifier i;
> expression E;
> position q != excluded.p;
> position p : script:python(i) { p[0].current_element == i};
> attribute name __randomize_layout;
> @@
>
> (
>  T i@q@p;
> |
>  T i@q@p=E;
> )
>
> @rr@
> type T;
> identifier i;
> expression E;
> attribute name __randomize_layout;
> @@
>
> (
>  T i;
> |
>  T i=E;
> )
>
> @script:python@
> i << excluded.i;
> @@
> print (i)
> ```
> It is a suboptimal version of Julia's last proposition.
> It happens that in a file a statement I expect to be matched is not detected.
> Troubleshooting the issue, I see a behavior that does not fit the model I have in my mind. This suggests to me that things (coccinelle under the hood mechanisms) are more complicated than I think.
> back to the point:
> Running the script and printing only the rule "rr", which is the same as "r" without positions, the results presents a set of entries containing the entry I'm interested in.
> Running the same script but printing only the rule "excluded", I see that the result is an empty set.
> Finally, running the script using only the "r" rule, the entry I'm interested in is not there.
> Surprisingly (for me), removing the position constraints coming from the rule "excluded" (position q != excluded.p;) from the rule "r", the entry appears.
> My question then is the following: How can the rule "excluded" that presents an empty set influence the rule "r"?
> Speculating Julia's words on a previous answer I got, I guessed that somehow an empty set could be treated differently, so I added a line into the target c file to make at least "excluded" match one line. But it didn't make any difference.

Sorry, but I can't answer a question at that abstract level.  Please send
the precise example (semntic patch and code), and I will check on what
went wrong.

julia

>
> Alessandro 
>
>
>
> Il giorno mar 10 mag 2022 alle ore 12:39 Julia Lawall <julia.lawall@inria.fr> ha scritto:
>       Here is a suggested solution for all of the problems reported so far:
>
>       @r@
>       type T;
>       identifier i;
>       expression E;
>       position p : script:python(i) { p[0].current_element == i };
>       attribute name __randomize_layout;
>       @@
>
>       (
>       T i(...);
>       |
>       T i(...,......);
>       |
>       extern T i;
>       |
>       T i@p;
>       |
>       T i@p=E;
>       )
>
>       @script:python@
>       i << r.i;
>       p << r.p;
>       @@
>       print (i)
>
>       ----------------------
>
>       The problem with __randomize_layout is that the Coccinelle parser is not
>       recognizing it as an attribute.  You can force that by putting a
>       declaration in the semantic patch.
>
>       It seems that ... in a function parameter list does not match the ... in C
>       for a variable list of arguments.  That ... can be matched explicitly by
>       ...... so I have added another case with that.
>
>       I als combined all of the patterns into one rule.  By inheriting the
>       position varaible p into the python rule at the end, that python rule is
>       only executed if p is defined, which makes it select the cases of interest
>       and ignore the others.
>
>       julia
>
>
>

Re: [cocci] List global variables with SmPL

[Alessandro Carminati]

[-- Attachment #1: Type: text/plain, Size: 4777 bytes --]

Sorry for this.
the script is:

```
@excluded@
type T;
identifier i;
position p;
@@

(
 T i@p(...);
|
 extern T i@p;
|
 T i@p(...,......);
)


@r@
type T;
identifier i;
expression E;
position q != excluded.p;
position p : script:python(i) { p[0].current_element == i};
attribute name __randomize_layout;
@@

(
 T i@q@p;
|
 T i@q@p=E;
)

@rr@
type T;
identifier i;
expression E;
attribute name __randomize_layout;
@@

(
 T i;
|
 T i=E;
)

@script:python@
i << r.i;
@@
print (i)
```
the target file is
https://raw.githubusercontent.com/torvalds/linux/master/sound/soc/mediatek/common/mtk-afe-platform-driver.c
and the symbol I'm looking for is mtk_afe_pcm_platform
I introduced the rule rr to attempt to debug the script. It is not meant to
be part of the final script.
Thank you for the help.

Alessandro


Il giorno mer 11 mag 2022 alle ore 09:44 Julia Lawall <julia.lawall@inria.fr>
ha scritto:

>
>
> On Wed, 11 May 2022, Alessandro Carminati wrote:
>
> > I'm planning to stop bugging this mailing list, but before that, please
> allow me a final question. That is related to what I have done so far.
> > My current implementation is the following.
> > ```
> > @excluded@
> > type T;
> > identifier i;
> > position p;
> > @@
> >
> > (
> >  T i@p(...);
> > |
> >  extern T i@p;
> > |
> >  T i@p(...,......);
> > )
> >
> >
> > @r@
> > type T;
> > identifier i;
> > expression E;
> > position q != excluded.p;
> > position p : script:python(i) { p[0].current_element == i};
> > attribute name __randomize_layout;
> > @@
> >
> > (
> >  T i@q@p;
> > |
> >  T i@q@p=E;
> > )
> >
> > @rr@
> > type T;
> > identifier i;
> > expression E;
> > attribute name __randomize_layout;
> > @@
> >
> > (
> >  T i;
> > |
> >  T i=E;
> > )
> >
> > @script:python@
> > i << excluded.i;
> > @@
> > print (i)
> > ```
> > It is a suboptimal version of Julia's last proposition.
> > It happens that in a file a statement I expect to be matched is not
> detected.
> > Troubleshooting the issue, I see a behavior that does not fit the model
> I have in my mind. This suggests to me that things (coccinelle under the
> hood mechanisms) are more complicated than I think.
> > back to the point:
> > Running the script and printing only the rule "rr", which is the same as
> "r" without positions, the results presents a set of entries containing the
> entry I'm interested in.
> > Running the same script but printing only the rule "excluded", I see
> that the result is an empty set.
> > Finally, running the script using only the "r" rule, the entry I'm
> interested in is not there.
> > Surprisingly (for me), removing the position constraints coming from the
> rule "excluded" (position q != excluded.p;) from the rule "r", the entry
> appears.
> > My question then is the following: How can the rule "excluded" that
> presents an empty set influence the rule "r"?
> > Speculating Julia's words on a previous answer I got, I guessed that
> somehow an empty set could be treated differently, so I added a line into
> the target c file to make at least "excluded" match one line. But it didn't
> make any difference.
>
> Sorry, but I can't answer a question at that abstract level.  Please send
> the precise example (semntic patch and code), and I will check on what
> went wrong.
>
> julia
>
> >
> > Alessandro
> >
> >
> >
> > Il giorno mar 10 mag 2022 alle ore 12:39 Julia Lawall <
> julia.lawall@inria.fr> ha scritto:
> >       Here is a suggested solution for all of the problems reported so
> far:
> >
> >       @r@
> >       type T;
> >       identifier i;
> >       expression E;
> >       position p : script:python(i) { p[0].current_element == i };
> >       attribute name __randomize_layout;
> >       @@
> >
> >       (
> >       T i(...);
> >       |
> >       T i(...,......);
> >       |
> >       extern T i;
> >       |
> >       T i@p;
> >       |
> >       T i@p=E;
> >       )
> >
> >       @script:python@
> >       i << r.i;
> >       p << r.p;
> >       @@
> >       print (i)
> >
> >       ----------------------
> >
> >       The problem with __randomize_layout is that the Coccinelle parser
> is not
> >       recognizing it as an attribute.  You can force that by putting a
> >       declaration in the semantic patch.
> >
> >       It seems that ... in a function parameter list does not match the
> ... in C
> >       for a variable list of arguments.  That ... can be matched
> explicitly by
> >       ...... so I have added another case with that.
> >
> >       I als combined all of the patterns into one rule.  By inheriting
> the
> >       position varaible p into the python rule at the end, that python
> rule is
> >       only executed if p is defined, which makes it select the cases of
> interest
> >       and ignore the others.
> >
> >       julia
> >
> >
> >

[-- Attachment #2: Type: text/html, Size: 6656 bytes --]

Re: [cocci] List global variables with SmPL

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1559 bytes --]

> It is a suboptimal version of Julia's last proposition.
> It happens that in a file a statement I expect to be matched is not detected.
> Troubleshooting the issue, I see a behavior that does not fit the model I have in my mind. This suggests to me that things (coccinelle under the hood mechanisms) are more complicated than I think.
> back to the point:
> Running the script and printing only the rule "rr", which is the same as "r" without positions, the results presents a set of entries containing the entry I'm interested in.
> Running the same script but printing only the rule "excluded", I see that the result is an empty set.
> Finally, running the script using only the "r" rule, the entry I'm interested in is not there.
> Surprisingly (for me), removing the position constraints coming from the rule "excluded" (position q != excluded.p;) from the rule "r", the entry appears.

You haven't provided tihs semantic patch, so I don't know in detail what
it does.

In my opinion, the rule excluded is completely irrelevant for this
example.  The code that you want to detect is:

const struct snd_soc_component_driver mtk_afe_pcm_platform = {
        .name           = AFE_PCM_NAME,
        .pointer        = mtk_afe_pcm_pointer,
	.pcm_construct  = mtk_afe_pcm_new,
};

{
        .name           = AFE_PCM_NAME,
        .pointer        = mtk_afe_pcm_pointer,
        .pcm_construct  = mtk_afe_pcm_new,
}

is not considered to be an ordinary expression, but rather as an
initializer.  Everything should work find if you just replace E by ...

julia

Re: [cocci] List global variables with SmPL

[Alessandro Carminati]

[-- Attachment #1: Type: text/plain, Size: 2597 bytes --]

Alessandro



Il giorno mer 11 mag 2022 alle ore 10:14 Julia Lawall <julia.lawall@inria.fr>
ha scritto:

> > It is a suboptimal version of Julia's last proposition.
> > It happens that in a file a statement I expect to be matched is not
> detected.
> > Troubleshooting the issue, I see a behavior that does not fit the model
> I have in my mind. This suggests to me that things (coccinelle under the
> hood mechanisms) are more complicated than I think.
> > back to the point:
> > Running the script and printing only the rule "rr", which is the same as
> "r" without positions, the results presents a set of entries containing the
> entry I'm interested in.
> > Running the same script but printing only the rule "excluded", I see
> that the result is an empty set.
> > Finally, running the script using only the "r" rule, the entry I'm
> interested in is not there.
> > Surprisingly (for me), removing the position constraints coming from the
> rule "excluded" (position q != excluded.p;) from the rule "r", the entry
> appears.
>
> You haven't provided tihs semantic patch, so I don't know in detail what
> it does.
>
It appears I do not know what a semantic patch is.
I was convinced that what I called the "script" included inline in the mail
I sent was the semantic patch.


> In my opinion, the rule excluded is completely irrelevant for this
> example.  The code that you want to detect is:
>
> const struct snd_soc_component_driver mtk_afe_pcm_platform = {
>         .name           = AFE_PCM_NAME,
>         .pointer        = mtk_afe_pcm_pointer,
>         .pcm_construct  = mtk_afe_pcm_new,
> };
>
> {
>         .name           = AFE_PCM_NAME,
>         .pointer        = mtk_afe_pcm_pointer,
>         .pcm_construct  = mtk_afe_pcm_new,
> }
>
> is not considered to be an ordinary expression, but rather as an
> initializer.  Everything should work find if you just replace E by ...
>

Indeed, replacing the E with ... solves the issue.
The explanation makes sense to me,  but why does the rule rr in the
following context matches mtk_afe_pcm_platform and presents the following
resulting set?
```
$ spatch  -sp_file  simple.cocci mtk-afe-platform-driver.c
init_defs_builtins: /usr/local/lib/coccinelle/standard.h
HANDLING: mtk-afe-platform-driver.c
afe
dai
dai_idx
dev
hw_base
hw_ptr
memif
memif_data
mtk_afe_pcm_platform
num_dai_drivers
pcm
pcm_ptr_bytes
reg_ofs_base
reg_ofs_cur
regmap
ret
rtd
size
```
and simple.cocci is
```
@rr@
type T;
identifier i;
expression E;
attribute name __randomize_layout;
@@

(
 T i;
|
 T i=E;
)

@script:python@
i << rr.i;
@@
print (i)
```


> julia

[-- Attachment #2: Type: text/html, Size: 3859 bytes --]

Re: [cocci] List global variables with SmPL

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 3906 bytes --]



On Wed, 11 May 2022, Alessandro Carminati wrote:

>
> Alessandro
>
>
>
> Il giorno mer 11 mag 2022 alle ore 10:14 Julia Lawall <julia.lawall@inria.fr> ha scritto:
>       > It is a suboptimal version of Julia's last proposition.
>       > It happens that in a file a statement I expect to be matched is not detected.
>       > Troubleshooting the issue, I see a behavior that does not fit the model I have in my mind. This suggests to me that things (coccinelle under the hood mechanisms) are more complicated than I think.
>       > back to the point:
>       > Running the script and printing only the rule "rr", which is the same as "r" without positions, the results presents a set of entries containing the entry I'm interested in.
>       > Running the same script but printing only the rule "excluded", I see that the result is an empty set.
>       > Finally, running the script using only the "r" rule, the entry I'm interested in is not there.
>       > Surprisingly (for me), removing the position constraints coming from the rule "excluded" (position q != excluded.p;) from the rule "r", the entry appears.
>
>       You haven't provided tihs semantic patch, so I don't know in detail what
>       it does.
>
> It appears I do not know what a semantic patch is.
> I was convinced that what I called the "script" included inline in the mail I sent was the semantic patch.

It is, but you said that you removed something.  Removing that thing is
not trivial because other things depend on it.  So I don't know exactly
what you did.

In your rr.cocci rule, an isomorphism applies to the pattern T i; to allow
it to match T i = ...; so the problem is solved.  This isomorphism does
not apply when there are position variables attached to the declared
identifier.  This information is provided when running spatch
--parse-cocci by the following message:

warning: iso decl_init does not match the code below on line 26
T i@p@q;

context metavariable Z is matched against the following
noncontext code:
i@p@q

You can see the definition of decl_init in the file standard.iso.

I'm not sure that this strategy for isomorphisms is correct.  At least in
this case, there seems to be no harm in considering i to be in a context
position.  It's not removed or added.  I'm not sure why having a position
metavariable disqualifies is.

julia

>
>
>       In my opinion, the rule excluded is completely irrelevant for this
>       example.  The code that you want to detect is:
>
>       const struct snd_soc_component_driver mtk_afe_pcm_platform = {
>               .name           = AFE_PCM_NAME,
>               .pointer        = mtk_afe_pcm_pointer,
>               .pcm_construct  = mtk_afe_pcm_new,
>       };
>
>       {
>               .name           = AFE_PCM_NAME,
>               .pointer        = mtk_afe_pcm_pointer,
>               .pcm_construct  = mtk_afe_pcm_new,
>       }
>
>       is not considered to be an ordinary expression, but rather as an
>       initializer.  Everything should work find if you just replace E by ...
>
>
> Indeed, replacing the E with ... solves the issue.
> The explanation makes sense to me,  but why does the rule rr in the following context matches mtk_afe_pcm_platform and presents the following resulting set?
> ```
> $ spatch  -sp_file  simple.cocci mtk-afe-platform-driver.c
> init_defs_builtins: /usr/local/lib/coccinelle/standard.h
> HANDLING: mtk-afe-platform-driver.c
> afe
> dai
> dai_idx
> dev
> hw_base
> hw_ptr
> memif
> memif_data
> mtk_afe_pcm_platform
> num_dai_drivers
> pcm
> pcm_ptr_bytes
> reg_ofs_base
> reg_ofs_cur
> regmap
> ret
> rtd
> size
> ```
> and simple.cocci is
> ```
> @rr@
> type T;
> identifier i;
> expression E;
> attribute name __randomize_layout;
> @@
>
> (
>  T i;
> |
>  T i=E;
> )
>
> @script:python@
> i << rr.i;
> @@
> print (i)
> ```
>
>
>       julia
>
>
>

Re: [cocci] List global variables with SmPL

[Markus Elfring]

[-- Attachment #1: Type: text/plain, Size: 506 bytes --]

> I was convinced that what I called the "script" included inline in the mail I sent was the semantic patch.


Such interpretation of the discussed SmPL code can be reasonable to some degree.

The term “patch” would usually be connected with some changes.
But you described so far that you would like to perform analyses for selected source code.
Thus modification specifications were omitted.

How do you think about to use the wording “semantic match” in this case?

Regards,
Markus

[-- Attachment #2: Type: text/html, Size: 991 bytes --]

Re: [cocci] List global variables with SmPL

[Markus Elfring]

> I'm planning to stop bugging this mailing list, …


Why do you think in such a direction at the moment?

Would you like to benefit from this communication interface any more?

Regards,
Markus

Re: [cocci] List global variables with SmPL

[Alessandro Carminati]

[-- Attachment #1: Type: text/plain, Size: 794 bytes --]

Good day, Markus.
This mailing list has been quite beneficial to me.
I learned a lot about coccinelle thanks to Julia, who helped me.
Still, I do not want to monopolize it for my own purposes.
So I put a halt to the discussion.
I have further intentions for coccinelle use in the future, which means
I'll probably bother you with different topics again. Coccinelle is a skill
I'd want to master.
For the time being, thank you all for your help.

Alessandro


Il giorno mer 11 mag 2022 alle ore 20:11 Markus Elfring <
Markus.Elfring@web.de> ha scritto:

> > I'm planning to stop bugging this mailing list, …
>
>
> Why do you think in such a direction at the moment?
>
> Would you like to benefit from this communication interface any more?
>
> Regards,
> Markus
>
>

[-- Attachment #2: Type: text/html, Size: 1357 bytes --]

Re: [cocci] List global variables with SmPL

[Markus Elfring]

> Still, I do not want to monopolize it for my own purposes.


I find that you do not need to be so concerned here.



> So I put a halt to the discussion.


Would such an action hinder evolution for desirable improvements?

Regards,
Markus