Re: [Cocci] How to match function calls in macros?

Re: [Cocci] How to match function calls in macros?

[Markus Elfring]

> @@
> identifier mac,f;
> @@
>
> *#define mac(...) <+... f(...) ...+>

Does such an analysis approach restrict the source code search really on
complete definitions for function-like macros?


Will it become interesting for clarify corresponding special cases?

* How are empty macros treated by the Coccinelle software?

* Is there a connection managed between a macro declaration
  and subsequent source code?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Julia Lawall]

On Mon, 18 May 2020, Markus Elfring wrote:

> > @@
> > identifier mac,f;
> > @@
> >
> > *#define mac(...) <+... f(...) ...+>
>
> Does such an analysis approach restrict the source code search really on
> complete definitions for function-like macros?

Coccinelle knows where the macro ends.  A macro has to be on one line,
perhaps extended with a \

julia


>
>
> Will it become interesting for clarify corresponding special cases?
>
> * How are empty macros treated by the Coccinelle software?
>
> * Is there a connection managed between a macro declaration
>   and subsequent source code?
>
> Regards,
> Markus
>
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Markus Elfring]

>>> @@
>>> identifier mac,f;
>>> @@
>>>
>>> *#define mac(...) <+... f(...) ...+>
>>
>> Does such an analysis approach restrict the source code search really on
>> complete definitions for function-like macros?
>
> Coccinelle knows where the macro ends.

This is good to know.


> A macro has to be on one line,

This fact is reasonable.


> perhaps extended with a \

How would (or should) the Coccinelle software handle the mentioned search pattern
if such a line continuation indication was omitted between a macro declaration
and subsequent source code?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Julia Lawall]

On Mon, 18 May 2020, Markus Elfring wrote:

> >>> @@
> >>> identifier mac,f;
> >>> @@
> >>>
> >>> *#define mac(...) <+... f(...) ...+>
> >>
> >> Does such an analysis approach restrict the source code search really on
> >> complete definitions for function-like macros?
> >
> > Coccinelle knows where the macro ends.
>
> This is good to know.
>
>
> > A macro has to be on one line,
>
> This fact is reasonable.
>
>
> > perhaps extended with a \
>
> How would (or should) the Coccinelle software handle the mentioned search pattern
> if such a line continuation indication was omitted between a macro declaration
> and subsequent source code?

The macro ends at the first line that doesn't end with a \

If the developer forgets the \ there would likely be a parsing problem.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Markus Elfring]

>> How would (or should) the Coccinelle software handle the mentioned search pattern
>> if such a line continuation indication was omitted between a macro declaration
>> and subsequent source code?
>
> The macro ends at the first line that doesn't end with a \

This is fine.


> If the developer forgets the \ there would likely be a parsing problem.

I find it helpful to clarify parsing challenges around such an use case.
It might occasionally be relevant if a line continuation indication
was accidentally forgotten or was intentionally omitted.

* Source code review should point such questionable places out, shouldn't it?

* How much does this implementation detail matter for the safe application
  of the semantic patch language?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Julia Lawall]

On Mon, 18 May 2020, Markus Elfring wrote:

> >> How would (or should) the Coccinelle software handle the mentioned search pattern
> >> if such a line continuation indication was omitted between a macro declaration
> >> and subsequent source code?
> >
> > The macro ends at the first line that doesn't end with a \
>
> This is fine.
>
>
> > If the developer forgets the \ there would likely be a parsing problem.
>
> I find it helpful to clarify parsing challenges around such an use case.
> It might occasionally be relevant if a line continuation indication
> was accidentally forgotten or was intentionally omitted.
>
> * Source code review should point such questionable places out, shouldn't it?
>
> * How much does this implementation detail matter for the safe application
>   of the semantic patch language?

This is not Coccinelle's problem.  A developer can run a compiler to check
for parsing errors.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Markus Elfring]

>>> If the developer forgets the \ there would likely be a parsing problem.
>>
>> I find it helpful to clarify parsing challenges around such an use case.
>> It might occasionally be relevant if a line continuation indication
>> was accidentally forgotten or was intentionally omitted.
>>
>> * Source code review should point such questionable places out, shouldn't it?
>>
>> * How much does this implementation detail matter for the safe application
>>   of the semantic patch language?
>
> This is not Coccinelle's problem.

This software is also involved then.


> A developer can run a compiler to check for parsing errors.

I imagine that a missing line continuation can be hard to detect
and report as a possible programming error because the corresponding
code parts can still be valid on their own according to special circumstances.

Can the semantic patch language help to insist for a search that a bit of source code
belongs to the implementation of a function-like macro?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Julia Lawall]

On Mon, 18 May 2020, Markus Elfring wrote:

> >>> If the developer forgets the \ there would likely be a parsing problem.
> >>
> >> I find it helpful to clarify parsing challenges around such an use case.
> >> It might occasionally be relevant if a line continuation indication
> >> was accidentally forgotten or was intentionally omitted.
> >>
> >> * Source code review should point such questionable places out, shouldn't it?
> >>
> >> * How much does this implementation detail matter for the safe application
> >>   of the semantic patch language?
> >
> > This is not Coccinelle's problem.
>
> This software is also involved then.
>
>
> > A developer can run a compiler to check for parsing errors.
>
> I imagine that a missing line continuation can be hard to detect
> and report as a possible programming error because the corresponding
> code parts can still be valid on their own according to special circumstances.

If they are valid, then what is the problem.  Neither a compiler nor
Coccinelle can know what the developer intended.

> Can the semantic patch language help to insist for a search that a bit
> of source code belongs to the implementation of a function-like macro?

That's what the search that was written does.  The pattern that comes
after #define has to be in the definition of the macro.  Coccinelle never
matches things in more than one top-level term in the C file.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Markus Elfring]

>> Can the semantic patch language help to insist for a search that a bit
>> of source code belongs to the implementation of a function-like macro?
>
> That's what the search that was written does.

I interpret the example in another way.

*#define mac(...) <+... f(...) ...+>


> The pattern that comes after #define has to be in the definition of the macro.

* Would you find it as acceptable if a function-like macro was implemented
  in the way that it became empty?

* If find that the specification “<+... f(...) ...+>” does not explicitly
  express a requirement for a replacement list (despite of being so close).
  https://en.cppreference.com/w/c/preprocessor/replace


> Coccinelle never matches things in more than one top-level term in the C file.

Which is the corresponding scope for such source code?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Markus Elfring]

>> Can the semantic patch language help to insist for a search that a bit
>> of source code belongs to the implementation of a function-like macro?
>
> That's what the search that was written does.  The pattern that comes
> after #define has to be in the definition of the macro.

I suggest to consider additional source code variants.
With which SmPL constructs should be ensured that a search pattern
like “<+... f(...) ...+>” refers only to content from the same logical source line?
https://en.cppreference.com/w/c/language/translation_phases#Phase_2

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]



On Tue, 19 May 2020, Markus Elfring wrote:

> >> Can the semantic patch language help to insist for a search that a bit
> >> of source code belongs to the implementation of a function-like macro?
> >
> > That's what the search that was written does.  The pattern that comes
> > after #define has to be in the definition of the macro.
>
> I suggest to consider additional source code variants.
> With which SmPL constructs should be ensured that a search pattern
> like “<+... f(...) ...+>” refers only to content from the same logical source line?
> https://en.cppreference.com/w/c/language/translation_phases#Phase_2

Please stop asking the same question over and over.  In the context of a
macro definition, <+... f(...) ...+> will only match what is from the same
logical source line.  Because that is all that there is in a macro
definition.  If Coccinelle is working on a macro definition, it works only
on that macro definition.  If it is working on a function definition, it
works only on that function definition.

julia

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Markus Elfring]

>> With which SmPL constructs should be ensured that a search pattern
>> like “<+... f(...) ...+>” refers only to content from the same logical source line?
>> https://en.cppreference.com/w/c/language/translation_phases#Phase_2
>
> Please stop asking the same question over and over.

I hope that the expected and actually supported software functionality
will become clearer.


> In the context of a macro definition, <+... f(...) ...+> will only match
> what is from the same logical source line.

Does the functionality of the SmPL nest construct change according to
previous source code?

Are there more aspects to consider for the safe handling of physical source lines?


> Because that is all that there is in a macro definition.

We hope so also according to the initial clarification request for this issue.


> If Coccinelle is working on a macro definition, it works only on that macro definition.

Such an expectation can eventually be fine.

Did I overlook any information for this detail in the software documentation so far?
https://github.com/coccinelle/coccinelle/blob/7cf2c23e64066d5249a64a316cc5347831f7a63f/docs/manual/cocci_syntax.tex#L1736


> If it is working on a function definition, it works only on that function definition.

Does the position of the define directive influence this view?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1507 bytes --]



On Tue, 19 May 2020, Markus Elfring wrote:

> >> With which SmPL constructs should be ensured that a search pattern
> >> like “<+... f(...) ...+>” refers only to content from the same logical source line?
> >> https://en.cppreference.com/w/c/language/translation_phases#Phase_2
> >
> > Please stop asking the same question over and over.
>
> I hope that the expected and actually supported software functionality
> will become clearer.
>
>
> > In the context of a macro definition, <+... f(...) ...+> will only match
> > what is from the same logical source line.
>
> Does the functionality of the SmPL nest construct change according to
> previous source code?
>
> Are there more aspects to consider for the safe handling of physical source lines?
>
>
> > Because that is all that there is in a macro definition.
>
> We hope so also according to the initial clarification request for this issue.
>
>
> > If Coccinelle is working on a macro definition, it works only on that macro definition.
>
> Such an expectation can eventually be fine.
>
> Did I overlook any information for this detail in the software documentation so far?
> https://github.com/coccinelle/coccinelle/blob/7cf2c23e64066d5249a64a316cc5347831f7a63f/docs/manual/cocci_syntax.tex#L1736
>
>
> > If it is working on a function definition, it works only on that function definition.
>
> Does the position of the define directive influence this view?

I believe that the #defines inside of other top-level definitions are
ignored.

julia

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Markus Elfring]

>>> If it is working on a function definition, it works only on that function definition.
>>
>> Does the position of the define directive influence this view?
>
> I believe that the #defines inside of other top-level definitions are ignored.

I find such a feedback surprising.

It seems that the clarification of desired software functionality can become
more interesting also with this information.
How will the supported software functionality evolve further
besides intra-procedural source code analysis?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Markus Elfring]

> If the developer forgets the \ there would likely be a parsing problem.

Another information can be helpful then.


elfring@Sonne:~/Projekte/Coccinelle/Probe> spatch --parse-c INIT_WORK-Test-20200518.c
…
SUSPICIOUS: a \ character appears outside of a #define at
INIT_WORK-Test-20200518.c:4


Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Markus Elfring]

> Normally, Coccinelle will match code inside of macro definitions,
> but only if it is able to parse the macro definition,

This is good to know.


> and the ability to parse macro definitions is somewhat limited.

How are the chances to reduce any software limitations in this area?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

[Cocci] How to match function calls in macros?

[Chuhong Yuan]

Hi all,
I want to write a script to match function calls in macros
but I don't know how to do that.
Here is an example:

#define __INIT_WORK(_work, _func, _onstack)     \
    do {     \
        static struct lock_class_key __key; \
        \
        __init_work((_work), _onstack); \
        (_work)->data = (atomic_long_t) WORK_DATA_INIT(); \
        lockdep_init_map(&(_work)->lockdep_map,
"(work_completion)"#_work, &__key, 0); \
        INIT_LIST_HEAD(&(_work)->entry); \
        (_work)->func = (_func); \
    } while (0)

In this example, I want to match function calls
like __init_work() and lockdep_init_map() in this macro.
So how to implement this by Coccinelle?

Thanks,
Chuhong
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Julia Lawall]

On Sat, 16 May 2020, Chuhong Yuan wrote:

> Hi all,
> I want to write a script to match function calls in macros
> but I don't know how to do that.
> Here is an example:
>
> #define __INIT_WORK(_work, _func, _onstack)     \
>     do {     \
>         static struct lock_class_key __key; \
>         \
>         __init_work((_work), _onstack); \
>         (_work)->data = (atomic_long_t) WORK_DATA_INIT(); \
>         lockdep_init_map(&(_work)->lockdep_map,
> "(work_completion)"#_work, &__key, 0); \
>         INIT_LIST_HEAD(&(_work)->entry); \
>         (_work)->func = (_func); \
>     } while (0)
>
> In this example, I want to match function calls
> like __init_work() and lockdep_init_map() in this macro.
> So how to implement this by Coccinelle?

I would suggest to take your file and run spatch --parse-c on the file.
If you find BAD or bad in front of the lines of this code then the problem
is that the code is not being parsed.  I suspect that the # is the
problem.

Normally, Coccinelle will match code inside of macro definitions, but only
if it is able to parse the macro definition, and the ability to parse
macro definitions is somewhat limited.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Chuhong Yuan]

On Sat, May 16, 2020 at 4:49 PM Julia Lawall <julia.lawall@inria.fr> wrote:
>
>
>
> On Sat, 16 May 2020, Chuhong Yuan wrote:
>
> > Hi all,
> > I want to write a script to match function calls in macros
> > but I don't know how to do that.
> > Here is an example:
> >
> > #define __INIT_WORK(_work, _func, _onstack)     \
> >     do {     \
> >         static struct lock_class_key __key; \
> >         \
> >         __init_work((_work), _onstack); \
> >         (_work)->data = (atomic_long_t) WORK_DATA_INIT(); \
> >         lockdep_init_map(&(_work)->lockdep_map,
> > "(work_completion)"#_work, &__key, 0); \
> >         INIT_LIST_HEAD(&(_work)->entry); \
> >         (_work)->func = (_func); \
> >     } while (0)
> >
> > In this example, I want to match function calls
> > like __init_work() and lockdep_init_map() in this macro.
> > So how to implement this by Coccinelle?
>
> I would suggest to take your file and run spatch --parse-c on the file.
> If you find BAD or bad in front of the lines of this code then the problem
> is that the code is not being parsed.  I suspect that the # is the
> problem.
>

I have run parse-c and it says the example file is perfect.

> Normally, Coccinelle will match code inside of macro definitions, but only
> if it is able to parse the macro definition, and the ability to parse
> macro definitions is somewhat limited.
>

My expression is not very clear.
I want to know which macros have function calls and what functions do they call.
So I wrote a script like this:

- #define mac(...) ... f(...) ...

But it does not work on the example.

> julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Julia Lawall]

On Sat, 16 May 2020, Chuhong Yuan wrote:

> On Sat, May 16, 2020 at 4:49 PM Julia Lawall <julia.lawall@inria.fr> wrote:
> >
> >
> >
> > On Sat, 16 May 2020, Chuhong Yuan wrote:
> >
> > > Hi all,
> > > I want to write a script to match function calls in macros
> > > but I don't know how to do that.
> > > Here is an example:
> > >
> > > #define __INIT_WORK(_work, _func, _onstack)     \
> > >     do {     \
> > >         static struct lock_class_key __key; \
> > >         \
> > >         __init_work((_work), _onstack); \
> > >         (_work)->data = (atomic_long_t) WORK_DATA_INIT(); \
> > >         lockdep_init_map(&(_work)->lockdep_map,
> > > "(work_completion)"#_work, &__key, 0); \
> > >         INIT_LIST_HEAD(&(_work)->entry); \
> > >         (_work)->func = (_func); \
> > >     } while (0)
> > >
> > > In this example, I want to match function calls
> > > like __init_work() and lockdep_init_map() in this macro.
> > > So how to implement this by Coccinelle?
> >
> > I would suggest to take your file and run spatch --parse-c on the file.
> > If you find BAD or bad in front of the lines of this code then the problem
> > is that the code is not being parsed.  I suspect that the # is the
> > problem.
> >
>
> I have run parse-c and it says the example file is perfect.
>
> > Normally, Coccinelle will match code inside of macro definitions, but only
> > if it is able to parse the macro definition, and the ability to parse
> > macro definitions is somewhat limited.
> >
>
> My expression is not very clear.
> I want to know which macros have function calls and what functions do they call.
> So I wrote a script like this:
>
> - #define mac(...) ... f(...) ...
>
> But it does not work on the example.

Try:

@@
identifier mac,f;
@@

*#define mac(...) <+... f(...) ...+>

... means that the code before and after the ... cannot be matched by the
... when considering control-flow paths.  Your macro is a loop, so there
is one call to f(...) after another. Coccinelle doesn't evaluate values,
so it doesn't realize that a while(0) won't loop.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to match function calls in macros?

[Chuhong Yuan]

On Sat, May 16, 2020 at 8:45 PM Julia Lawall <julia.lawall@inria.fr> wrote:
>
>
>
> On Sat, 16 May 2020, Chuhong Yuan wrote:
>
> > On Sat, May 16, 2020 at 4:49 PM Julia Lawall <julia.lawall@inria.fr> wrote:
> > >
> > >
> > >
> > > On Sat, 16 May 2020, Chuhong Yuan wrote:
> > >
> > > > Hi all,
> > > > I want to write a script to match function calls in macros
> > > > but I don't know how to do that.
> > > > Here is an example:
> > > >
> > > > #define __INIT_WORK(_work, _func, _onstack)     \
> > > >     do {     \
> > > >         static struct lock_class_key __key; \
> > > >         \
> > > >         __init_work((_work), _onstack); \
> > > >         (_work)->data = (atomic_long_t) WORK_DATA_INIT(); \
> > > >         lockdep_init_map(&(_work)->lockdep_map,
> > > > "(work_completion)"#_work, &__key, 0); \
> > > >         INIT_LIST_HEAD(&(_work)->entry); \
> > > >         (_work)->func = (_func); \
> > > >     } while (0)
> > > >
> > > > In this example, I want to match function calls
> > > > like __init_work() and lockdep_init_map() in this macro.
> > > > So how to implement this by Coccinelle?
> > >
> > > I would suggest to take your file and run spatch --parse-c on the file.
> > > If you find BAD or bad in front of the lines of this code then the problem
> > > is that the code is not being parsed.  I suspect that the # is the
> > > problem.
> > >
> >
> > I have run parse-c and it says the example file is perfect.
> >
> > > Normally, Coccinelle will match code inside of macro definitions, but only
> > > if it is able to parse the macro definition, and the ability to parse
> > > macro definitions is somewhat limited.
> > >
> >
> > My expression is not very clear.
> > I want to know which macros have function calls and what functions do they call.
> > So I wrote a script like this:
> >
> > - #define mac(...) ... f(...) ...
> >
> > But it does not work on the example.
>
> Try:
>
> @@
> identifier mac,f;
> @@
>
> *#define mac(...) <+... f(...) ...+>
>
> ... means that the code before and after the ... cannot be matched by the
> ... when considering control-flow paths.  Your macro is a loop, so there
> is one call to f(...) after another. Coccinelle doesn't evaluate values,
> so it doesn't realize that a while(0) won't loop.
>

This works, thank you!

> julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci