[Cocci] accessing comments

[Cocci] accessing comments

[Julia Lawall]

It is now possible to match the comments before, within, and after the
match of a token, via a comments metavariable.  An example is shown in
demos/comments.{c,cocci}.  Comments metavariables are currently only
accessible in OCaml script code, not python script code.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> It is now possible to match the comments before, within, and after the
> match of a token, via a comments metavariable.

Are there any structures supported within source code comments?

How do you think about to filter any text by regular expressions
for semantic patch language constraints?

Would you like to extend your software further so that such functionality
will work together also with SmPL asterisks?
https://github.com/coccinelle/coccinelle/issues/57

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Julia Lawall]

On Thu, 23 May 2019, Markus Elfring wrote:

> > It is now possible to match the comments before, within, and after the
> > match of a token, via a comments metavariable.
>
> Are there any structures supported within source code comments?

You get a list of strings, with one comment per string.

>
> How do you think about to filter any text by regular expressions
> for semantic patch language constraints?

Not supported.

> Would you like to extend your software further so that such functionality
> will work together also with SmPL asterisks?
> https://github.com/coccinelle/coccinelle/issues/57

No idea what you want here.  The only thing you can do is collect the
comments that are before, within, or after the match of a token.  There
are no constraints.  There is no way to specify what comments should be
there.  I don't think * makes any sense in this case.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> How do you think about to filter any text by regular expressions
>> for semantic patch language constraints?
>
> Not supported.

Would the text filter be possible by using (OCaml) script rules so far?

How will these limitations be represented in the software documentation?


>> Would you like to extend your software further so that such functionality
>> will work together also with SmPL asterisks?
>> https://github.com/coccinelle/coccinelle/issues/57
>
> No idea what you want here.

I suggest to show selected contents directly (similar to other metavariable types).

@display@
comments C;
@@
*C


> The only thing you can do is collect the comments that are before,
> within, or after the match of a token.

Under which circumstances will adjustments become supported there?


> There are no constraints.

Will this situation be reconsidered?


> There is no way to specify what comments should be there.

I hope that this will become possible.


> I don't think * makes any sense in this case.

I have got an other software usage expectation here.
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Julia Lawall]

On Thu, 23 May 2019, Markus Elfring wrote:

> >> How do you think about to filter any text by regular expressions
> >> for semantic patch language constraints?
> >
> > Not supported.
>
> Would the text filter be possible by using (OCaml) script rules so far?

You can do whatever you want in the ocaml code.

>
> How will these limitations be represented in the software documentation?
>
>
> >> Would you like to extend your software further so that such functionality
> >> will work together also with SmPL asterisks?
> >> https://github.com/coccinelle/coccinelle/issues/57
> >
> > No idea what you want here.
>
> I suggest to show selected contents directly (similar to other metavariable types).
>
> @display@
> comments C;
> @@
> *C

This is not possible.  Comments metavariables can only be attached to
other tokens.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> @display@
>> comments C;
>> @@
>> *C
>
> This is not possible.

At the moment.


> Comments metavariables can only be attached to other tokens.

I find that his wording can be ambiguous.

How do you think about views around literate programming
where executable code is between these comments?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Development challenges around OCaml usage for SmPL

[Markus Elfring]

> You can do whatever you want in the ocaml code.

This programming language has got also a general potential.
But I find that there are special development challenges to consider
for software components which do not belong to its standard library.
How do you think about to take another look at an extension request
like “Support for specification of additional library directories”
for the semantic patch language?
https://github.com/coccinelle/coccinelle/issues/19

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Checking configuration challenges around OCaml usage for SmPL

[Markus Elfring]

> You can do whatever you want in the ocaml code.

How will the following software situation be resolved?

elfring@Sonne:~/Projekte/Coccinelle/20160205> COCCINELLE_HOME=$(pwd) /usr/local/bin/opam config exec --switch 4.07.1 ./spatch demos/comments.cocci demos/comments.c
init_defs_builtins: /home/elfring/Projekte/Coccinelle/20160205/standard.h
Using native version of ocamlc/ocamlopt/ocamldep
ocamlc.opt -c /tmp/ocaml_cocci_bbc38d.cmo -g -I /home/elfring/Projekte/Coccinelle/20160205/ocaml  -I /usr/lib64/ocaml /tmp/ocaml_cocci_bbc38d.ml
File "/tmp/ocaml_cocci_bbc38d.ml", line 1:
Error: /usr/lib64/ocaml/list.cmi
is not a compiled interface for this version of OCaml.
It seems to be for an older version of OCaml.
Fatal error: exception Coccinelle_modules.Yes_prepare_ocamlcocci.CompileFailure("/tmp/ocaml_cocci_bbc38d.ml")


Can the dependency on the system compiler version be avoided here?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Checking configuration challenges around OCaml usage for SmPL

[Julia Lawall]

On Fri, 24 May 2019, Markus Elfring wrote:

> > You can do whatever you want in the ocaml code.
>
> How will the following software situation be resolved?

make distclean
./autogen
./configure
make

>
> elfring@Sonne:~/Projekte/Coccinelle/20160205> COCCINELLE_HOME=$(pwd) /usr/local/bin/opam config exec --switch 4.07.1 ./spatch demos/comments.cocci demos/comments.c
> init_defs_builtins: /home/elfring/Projekte/Coccinelle/20160205/standard.h
> Using native version of ocamlc/ocamlopt/ocamldep
> ocamlc.opt -c /tmp/ocaml_cocci_bbc38d.cmo -g -I /home/elfring/Projekte/Coccinelle/20160205/ocaml  -I /usr/lib64/ocaml /tmp/ocaml_cocci_bbc38d.ml
> File "/tmp/ocaml_cocci_bbc38d.ml", line 1:
> Error: /usr/lib64/ocaml/list.cmi
> is not a compiled interface for this version of OCaml.
> It seems to be for an older version of OCaml.
> Fatal error: exception Coccinelle_modules.Yes_prepare_ocamlcocci.CompileFailure("/tmp/ocaml_cocci_bbc38d.ml")
>
>
> Can the dependency on the system compiler version be avoided here?

There is no dependency on any particular system compiler version.  You
just seem to have compiled some of your code, then upgraded your ocaml,
then tried to compile the rest.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Checking configuration challenges around OCaml usage for SmPL

[Markus Elfring]

>> How will the following software situation be resolved?
>
> make distclean
> ./autogen
> ./configure
> make

These commands were executed also on my system yesterday.


> There is no dependency on any particular system compiler version.

Why is the program “ocamlc.opt” looking for the file “/usr/lib64/ocaml/list.cmi”
instead of using the selected variant “~/.opam/4.07.1/lib/ocaml/stdlib__list.cmi”?


> You just seem to have compiled some of your code, then upgraded your ocaml,
> then tried to compile the rest.

I find this view inappropriate when the command “opam config exec --switch 4.07.1”
should be relevant here.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Checking configuration challenges around OCaml usage for SmPL

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 853 bytes --]



On Fri, 24 May 2019, Markus Elfring wrote:

> >> How will the following software situation be resolved?
> >
> > make distclean
> > ./autogen
> > ./configure
> > make
>
> These commands were executed also on my system yesterday.
>
>
> > There is no dependency on any particular system compiler version.
>
> Why is the program “ocamlc.opt” looking for the file “/usr/lib64/ocaml/list.cmi”
> instead of using the selected variant “~/.opam/4.07.1/lib/ocaml/stdlib__list.cmi”?

OK, I don't know.  Either Thierry knows or there is some problem with the
setup of your system.

julia


>
>
> > You just seem to have compiled some of your code, then upgraded your ocaml,
> > then tried to compile the rest.
>
> I find this view inappropriate when the command “opam config exec --switch 4.07.1”
> should be relevant here.
>
> Regards,
> Markus
>

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Checking configuration challenges around OCaml usage for SmPL

[Markus Elfring]

>> Why is the program “ocamlc.opt” looking for the file “/usr/lib64/ocaml/list.cmi”
>> instead of using the selected variant “~/.opam/4.07.1/lib/ocaml/stdlib__list.cmi”?
>
> OK, I don't know.  Either Thierry knows or there is some problem with the
> setup of your system.

Which test results do you get for a command like “opam config exec --switch 4.07.1”?

Do you occasionally work also with different versions of software components
from OCaml's standard library?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Checking configuration challenges around OCaml usage for SmPL

[Markus Elfring]

>> ocamlc.opt -c /tmp/ocaml_cocci_bbc38d.cmo -g -I /home/elfring/Projekte/Coccinelle/20160205/ocaml  -I /usr/lib64/ocaml /tmp/ocaml_cocci_bbc38d.ml

Do you know where the include parameters are determined for such a call
of the OCaml compiler in your source code?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Checking configuration challenges around OCaml usage for SmPL

[Julia Lawall]

On Fri, 24 May 2019, Markus Elfring wrote:

> >> ocamlc.opt -c /tmp/ocaml_cocci_bbc38d.cmo -g -I /home/elfring/Projekte/Coccinelle/20160205/ocaml  -I /usr/lib64/ocaml /tmp/ocaml_cocci_bbc38d.ml
>
> Do you know where the include parameters are determined for such a call
> of the OCaml compiler in your source code?

I had no trouble compiling with ocaml 4.07.1

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Checking configuration challenges around OCaml usage for SmPL

[Markus Elfring]

>>>> ocamlc.opt -c /tmp/ocaml_cocci_bbc38d.cmo -g -I /home/elfring/Projekte/Coccinelle/20160205/ocaml  -I /usr/lib64/ocaml /tmp/ocaml_cocci_bbc38d.ml
>>
>> Do you know where the include parameters are determined for such a call
>> of the OCaml compiler in your source code?
>
> I had no trouble compiling with ocaml 4.07.1

Is your system compiler version accidentally the same in comparison to
the mentioned OPAM command example?

Should the implementation of the function “dep_flag” be improved
so that the variable “inc” in the function “load_file” for
my system configuration?
https://github.com/coccinelle/coccinelle/blob/b5c8841872ddd04815e9499863c8f9e6fda56fab/ocaml/yes_prepare_ocamlcocci.ml#L533

Will any more software evolution be better taken into account
around the usage of the program “ocamldep”?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Checking configuration challenges around OCaml usage for SmPL

[Markus Elfring]

>>>> ocamlc.opt -c /tmp/ocaml_cocci_bbc38d.cmo -g -I /home/elfring/Projekte/Coccinelle/20160205/ocaml  -I /usr/lib64/ocaml /tmp/ocaml_cocci_bbc38d.ml
>>
>> Do you know where the include parameters are determined for such a call
>> of the OCaml compiler in your source code?
>
> I had no trouble compiling with ocaml 4.07.1

Would you like to reconsider the usage of the variable “empty_deps”
in the implementation of function “parse_dep”?
https://github.com/coccinelle/coccinelle/blob/b5c8841872ddd04815e9499863c8f9e6fda56fab/ocaml/yes_prepare_ocamlcocci.ml#L489

How should other file names be determined for the compiled OCaml
interface descriptions (like “~/.opam/4.07.1/lib/ocaml/stdlib__list.cmi”)?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Changing include parameters for compilation of OCaml code from SmPL scripts

[Markus Elfring]

>> ocamlc.opt -c /tmp/ocaml_cocci_bbc38d.cmo -g -I /home/elfring/Projekte/Coccinelle/20160205/ocaml  -I /usr/lib64/ocaml /tmp/ocaml_cocci_bbc38d.ml

How can it be achieved to replace the shown reference to a directory
of the (standard) library for the system OCaml compiler?


elfring@Sonne:~> /usr/local/bin/opam config exec --switch 4.07.1 ocamlfind printconf stdlib
/usr/lib64/ocaml

Does such an information need further development considerations?


Would you like to be able to specify other include parameters
for the used compilation command?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> You get a list of strings, with one comment per string.

It seems that this metavariable provides structured information
which can be split according to your OCaml code in the SmPL script.

“…
let (c1b,c1m,c1a) = List.hd c1 in
…”


Can this programming approach result in undesirable data duplication?


I have tried it out after I replaced a placeholder “%s” by the
code “\"${OPAM_SWITCH_PREFIX}/ocaml\"” in the function “load_file”.
https://github.com/coccinelle/coccinelle/blob/b5c8841872ddd04815e9499863c8f9e6fda56fab/ocaml/yes_prepare_ocamlcocci.ml#L572

elfring@Sonne:~/Projekte/Coccinelle/20160205> COCCINELLE_HOME=$(pwd) /usr/local/bin/opam config exec --switch 4.07.1 ./spatch.opt demos/comments.cocci demos/comments.c
…
c1b: /* before the function …  */
…


* Now I wonder why this display is presented in three variations.

* Which texts would you expect for your variables “…m”?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> “…
>> let (c1b,c1m,c1a) = List.hd c1 in
>> …”
>>
>>
>> Can this programming approach result in undesirable data duplication?
>
> There is no data duplication.

The involved data structures are unclear at the moment.


> Learn about pointers.

How many different pointers will refer to empty strings here?



>> * Which texts would you expect for your variables “…m”?
>
> There is an example:
>
> c4b: /* a comment1 */
> c4m: /*in arglist*/
> c4a: /* a comment2 */

How would you like to handle source code elements without
parameter lists?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 825 bytes --]



On Sat, 25 May 2019, Markus Elfring wrote:

> >> “…
> >> let (c1b,c1m,c1a) = List.hd c1 in
> >> …”
> >>
> >>
> >> Can this programming approach result in undesirable data duplication?
> >
> > There is no data duplication.
>
> The involved data structures are unclear at the moment.
>
>
> > Learn about pointers.
>
> How many different pointers will refer to empty strings here?
>
>
>
> >> * Which texts would you expect for your variables “…m”?
> >
> > There is an example:
> >
> > c4b: /* a comment1 */
> > c4m: /*in arglist*/
> > c4a: /* a comment2 */
>
> How would you like to handle source code elements without
> parameter lists?

Parameter lists are irrelevant.  That is just an example.  As long as the
SmPL token matches more than one token, it is possible that there is a
comment between them.

julia

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> Parameter lists are irrelevant.  That is just an example.

We have got a different understanding for the desired comment handling
there at the moment.


> As long as the SmPL token matches more than one token,

Would we like to distinguish and clarify this case any further?


> it is possible that there is a comment between them.

This software aspect can be reasonable.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Julia Lawall]

On Sun, 26 May 2019, Markus Elfring wrote:

> > Parameter lists are irrelevant.  That is just an example.
>
> We have got a different understanding for the desired comment handling
> there at the moment.
>
>
> > As long as the SmPL token matches more than one token,
>
> Would we like to distinguish and clarify this case any further?

You can attach a comment metavariable to any token in the semantic patch.
some of these are metavariables.  Metavariables (except identifier
metavariables) often match more than one token in the C code.

julia

>
>
> > it is possible that there is a comment between them.
>
> This software aspect can be reasonable.
>
> Regards,
> Markus
>
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> You can attach a comment metavariable to any token in the semantic patch.
> some of these are metavariables.  Metavariables (except identifier
> metavariables) often match more than one token in the C code.

Can this case distinction trigger corresponding software design adjustments?

How often will it matter if only one item is involved instead of
several places?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> How often will it matter if only one item is involved instead of
>> several places?
>
> Your approach introduces pointless complexity to distinguish between th
> different cases.

I got an other software development opinion.
It might take another while to agree on useful case distinctions.

Do any items exist which can not contain comment data so that an attribute
like “within” can occasionally be avoided?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> You can attach a comment metavariable to any token in the semantic patch.
> some of these are metavariables.  Metavariables (except identifier
> metavariables) often match more than one token in the C code.

Can this case distinction trigger corresponding software design adjustments?

How often will it matter if only one item is involved instead of
several places?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 514 bytes --]



On Sat, 25 May 2019, Markus Elfring wrote:

> >> “…
> >> let (c1b,c1m,c1a) = List.hd c1 in
> >> …”
> >>
> >>
> >> Can this programming approach result in undesirable data duplication?
> >
> > There is no data duplication.
>
> The involved data structures are unclear at the moment.
>
>
> > Learn about pointers.
>
> How many different pointers will refer to empty strings here?

None.  If there is no comment, it will be the empty list.  The before,
within and after comments are lists of strings.

julia

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> How many different pointers will refer to empty strings here?
>
> None.  If there is no comment, it will be the empty list.

How many different pointers can occur for these empty items?


> The before, within and after comments are lists of strings.

Will data structures become helpful where less than three attributes
would be provided because a special source code element would be selected?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> Will data structures become helpful where less than three attributes
>> would be provided because a special source code element would be selected?
>
> Stop the pointless micro optimization.  This has no impact on anything.

We got different programming views which influence another approach
to achieve a better common understanding also for this software area.
I guess that the situation can be improved on both sides.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> The before, within and after comments are lists of strings.

I got further software development ideas around the provided data structures.

* If you would like to search within the text of multi-line comments
  by the means of regular expressions, the list elements need to be
  concatenated before.
  See also:
  Using a match result from an OCaml rule as a SmPL dependency
  https://systeme.lip6.fr/pipermail/cocci/2019-May/005889.html

  I imagine then that it would occasionally be more convenient to work with
  a string directly instead of a list.

* Advanced source code analysis and transformation triggers the need
  to manage the provided data exactly. The determination of source code positions
  is required then for safe data processing.
  Should the programming interface be a class then (which is different from a list)?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> * Now I wonder why this display is presented in three variations.
>
> Before within and after

I find this software behaviour hard to understand at the moment
when the shown metavariables were split into three parts already
by the call of the OCaml function “List.hd”.

Which comment would be provided as “within” text?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> There is one in an argument list that is within a statement.

This would only matter for the second call of the function “foo”
in the example function “main”, wouldn't it?
https://github.com/coccinelle/coccinelle/blob/210ce894d2dd1572fff9e1c98ae443e6df14f4c7/demos/comments.c#L2


How does your software decide which comment should be treated
as simple text before or after an item?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 837 bytes --]



On Sat, 25 May 2019, Markus Elfring wrote:

> > There is one in an argument list that is within a statement.
>
> This would only matter for the second call of the function “foo”
> in the example function “main”, wouldn't it?
> https://github.com/coccinelle/coccinelle/blob/210ce894d2dd1572fff9e1c98ae443e6df14f4c7/demos/comments.c#L2
>
>
> How does your software decide which comment should be treated
> as simple text before or after an item?

It takes all of the comments before (up to the preceding non comment
token if any) within and after (up to the following non comment token if
any).

I'm not sure what happens at the end of a function.  I suspect that the
end of a function is the end.  The comments that are before the start of
the next function are not considered to be after the function that
precedes them.

julia

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> How does your software decide which comment should be treated
>> as simple text before or after an item?
>
> It takes all of the comments before (up to the preceding non comment
> token if any) within and after (up to the following non comment token
> if any).

I find that this information can be confusing.
Will the desired text relationships become clearer?


> I'm not sure what happens at the end of a function.

Interesting special case.


> I suspect that the end of a function is the end.  The comments that
> are before the start of the next function are not considered to be after
> the function that precedes them.

Will such details become relevant also for the SmPL manual?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> There is one in an argument list that is within a statement.

Can it be that such a comment would belong to the function call operator
(open or closing parenthesis) instead of the identifier “foo”?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 471 bytes --]



On Sat, 25 May 2019, Markus Elfring wrote:

> > There is one in an argument list that is within a statement.
>
> Can it be that such a comment would belong to the function call operator
> (open or closing parenthesis) instead of the identifier “foo”?

You put the comment metavariable on whatever you want.  The report for the
comment in the argument list comes from the comment metavariable on the
complete statement, not on the comment metavariable on foo.

julia

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> Can it be that such a comment would belong to the function call operator
>> (open or closing parenthesis) instead of the identifier “foo”?
>
> You put the comment metavariable on whatever you want.

This functionality is generally nice.


> The report for the comment in the argument list comes from the comment metavariable
> on the complete statement, not on the comment metavariable on foo.

Thanks for such feedback.

It seems that this information can trigger expectations for further clarification
of the desired software behaviour.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> elfring@Sonne:~/Projekte/Coccinelle/20160205> COCCINELLE_HOME=$(pwd) /usr/local/bin/opam config exec --switch 4.07.1 ./spatch.opt demos/comments.cocci demos/comments.c
>> …
>> c1b: /* before the function …  */
>> …
>>
>>
>> * Now I wonder why this display is presented in three variations.
>
> Before within and after

I assume that this terse information can indicate another temporary
communication difficulty.
So I try once more to achieve a corresponding clarification also
in this area.

* How do you think about my expectation that the OCaml script rule
  should be evaluated only once because a single function implementation
  example is analysed here?

* Can the determination which text should belong to the attribute
  “before” or “after” become more challenging for statement metavariables?


>> * Which texts would you expect for your variables “…m”?

Can fragments of identifiers be mixed with comment data?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

>> * How do you think about my expectation that the OCaml script rule
>>   should be evaluated only once because a single function implementation
>>   example is analysed here?
>
> The multiple times have to do with the ... not the comment variables.

I do not expect so far that the SmPL code “... when any” should trigger three displays
for the shown test example.


>> * Can the determination which text should belong to the attribute
>>   “before” or “after” become more challenging for statement metavariables?
>
> We know what tokens are matched by the metavariable.  The comments are attached to the tokens.

How is the situation treated that a comment is between two executable statements?


> It is then straightforward to make the three lists.

I do not directly follow this view at the moment.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> The when any's allow any possible match.

I agree to this technical aspect.


> There are three calls to foo so three possible matches.

I find that this possibility does not require to repeat the evaluation
for the complete OCaml script rule.
I would find a possible repetition eventually reasonable for the SmPL code
between the ellipses.
But I would interpret the code “\(foo@c3();\&S@c4\)” in the way
that it should match only one function call with the expected comment combination
in this test finally.


>> How is the situation treated that a comment is between two executable statements?
>
> What is between two statements is after the first one

There is an usual before-after relationship.


> sad before the second one.

  ???


> Any comments will be in the after list of the first
> and the before list of the second.

Does this feedback mean that such a comment should identically appear
in two lists from different attributes?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> The when any's allow any possible match.  There are three calls to foo so three possible matches.

Can my software interpretation of unexpected code repetition be adjusted
also by moving the code “\(foo@c3();\&S@c4\)” into another SmPL rule?
Can it make sense to split the OCaml script rule then too?


Did I expect a bit too much here to filter the source code for existing comments
(instead of accepting also the usual possibility for omitted comments)?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Setting the match result by SmPL script rule

[Markus Elfring]

>> How do you think about to filter any text by regular expressions
>> for semantic patch language constraints?
>
> Not supported.

This software limitation and current development status has got consequences
as usual.

* It would be nice if comment contents could be directly restricted also by
  the means of SmPL constraints.
  I am unsure about the chances for extensions in this area.

* The supported languages “OCaml” and “Python” offer known programming interfaces
  for the work with regular expressions.
  The evaluation of these regexes can trigger useful data processing.
  But I am looking for ways now to set the match result for SmPL script rules.
  How are the chances to extend Coccinelle's software library for this purpose?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] accessing comments

[Markus Elfring]

> It is now possible to match the comments before, within, and after the
> match of a token, via a comments metavariable.

Would anybody become interested to discuss corresponding software development
challenges and fine-tuning possibilities any more since the support for
the Python programming language was added today?
https://systeme.lip6.fr/pipermail/cocci/2019-June/005934.html
https://lore.kernel.org/cocci/alpine.DEB.2.21.1906021038020.10641@hadrien/

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci