[Cocci] how to replace obsolete #ifdef's

[Cocci] how to replace obsolete #ifdef's

[Enrico Weigelt, metux IT consult]

Hi folks,


is there a way to replace obsolete #ifdef's via spatch ?

Here's my scenario:

In the linux kernel we have many places where drivers statically assign
pointers to match tables (eg. for oftree or acpi) into a global driver
struct - sometimes this is enclosed into some #ifdef CONFIG_..., when
things like oftree or acpi are optional.

Example patch for illustrating what I'd like to do:

--- a/sound/soc/codecs/pcm3060-i2c.c
+++ b/sound/soc/codecs/pcm3060-i2c.c
@@ -45,9 +45,7 @@ static int pcm3060_i2c_probe(struct i2c_client *i2c,
  static struct i2c_driver pcm3060_i2c_driver = {
         .driver = {
                 .name = "pcm3060",
-#ifdef CONFIG_OF
-               .of_match_table = pcm3060_of_match,
-#endif /* CONFIG_OF */
+               .of_match_table = of_match_ptr(pcm3060_of_match),
         },
         .id_table = pcm3060_i2c_id,
         .probe = pcm3060_i2c_probe,


Just wrapping the assignment into the macro call turned out to be easy.
But I haven't found a way to remove the now #ifdef :(

Any idea how I could solve this ?


--mtx

-- 
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Markus Elfring]

> Just wrapping the assignment into the macro call turned out to be easy.

This feedback seems to be promising.


> But I haven't found a way to remove the now #ifdef :(

I get the following test result for a corresponding SmPL transformation approach.

@replacement@
identifier value;
@@
-#ifdef CONFIG_OF
 .of_match_table =
-                  value
+                  of_match_ptr(value)
 ,
-#endif /* CONFIG_OF */


elfring@Sonne:~/Projekte/Linux/next-patched> spatch ~/Projekte/Coccinelle/Probe/Weigelt1.cocci sound/soc/codecs/pcm3060-i2c.c
init_defs_builtins: /usr/local/bin/../lib/coccinelle/standard.h
File "/home/elfring/Projekte/Coccinelle/Probe/Weigelt1.cocci", line 4, column 1, charpos = 36
  around = '#ifdef CONFIG_OF',
  whole content = -#ifdef CONFIG_OF


> Any idea how I could solve this ?

How are the chances for contribution of additional software development resources
according to the information “Support for matching #ifdef, etc.”?
https://github.com/coccinelle/coccinelle/blob/19ee1697bf152d37a78a20cefe148775bf4b0e0d/changes.txt#upcomingplannedtodo

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] how to replace obsolete #ifdef's

[Julia Lawall]

On Wed, 5 Jun 2019, Enrico Weigelt, metux IT consult wrote:

> Hi folks,
>
>
> is there a way to replace obsolete #ifdef's via spatch ?
>
> Here's my scenario:
>
> In the linux kernel we have many places where drivers statically assign
> pointers to match tables (eg. for oftree or acpi) into a global driver
> struct - sometimes this is enclosed into some #ifdef CONFIG_..., when
> things like oftree or acpi are optional.
>
> Example patch for illustrating what I'd like to do:
>
> --- a/sound/soc/codecs/pcm3060-i2c.c
> +++ b/sound/soc/codecs/pcm3060-i2c.c
> @@ -45,9 +45,7 @@ static int pcm3060_i2c_probe(struct i2c_client *i2c,
>  static struct i2c_driver pcm3060_i2c_driver = {
>         .driver = {
>                 .name = "pcm3060",
> -#ifdef CONFIG_OF
> -               .of_match_table = pcm3060_of_match,
> -#endif /* CONFIG_OF */
> +               .of_match_table = of_match_ptr(pcm3060_of_match),
>         },
>         .id_table = pcm3060_i2c_id,
>         .probe = pcm3060_i2c_probe,
>
>
> Just wrapping the assignment into the macro call turned out to be easy.
> But I haven't found a way to remove the now #ifdef :(

In principle you can remove some initializations and add them back.  But
doing so would have the unfortunate side effect of removing the newlines.
So I am trying to fix that.

julia


>
> Any idea how I could solve this ?
>
>
> --mtx
>
> --
> Enrico Weigelt, metux IT consult
> Free software and Linux embedded engineering
> info@metux.net -- +49-151-27565287
> _______________________________________________
> Cocci mailing list
> Cocci@systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] how to replace obsolete #ifdef's

[Enrico Weigelt, metux IT consult]

On 05.06.19 20:52, Julia Lawall wrote:

Hi,

> In principle you can remove some initializations and add them back.  

How can I match/remote on that "#ifdef ..." ?

Tried that, but got similar errors like Markus got (see his recent
mail).

It seems that spatch currently just doesn't understand preprocessor
directives at all, just treats them as literal strings. Maybe that
even would be fine in my case, if I only could match on that.

Any way for matching just a raw text pattern (w/o being parsed),
which includes special chars (eg. #) ?

--mtx

-- 
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] how to replace obsolete #ifdef's

[Julia Lawall]

On Wed, 5 Jun 2019, Enrico Weigelt, metux IT consult wrote:

> On 05.06.19 20:52, Julia Lawall wrote:
>
> Hi,
>
> > In principle you can remove some initializations and add them back.
>
> How can I match/remote on that "#ifdef ..." ?
>
> Tried that, but got similar errors like Markus got (see his recent
> mail).
>
> It seems that spatch currently just doesn't understand preprocessor
> directives at all, just treats them as literal strings. Maybe that
> even would be fine in my case, if I only could match on that.
>
> Any way for matching just a raw text pattern (w/o being parsed),
> which includes special chars (eg. #) ?

#ifdefs are comments.  If you remove a contiguous sequence of things, the
comments between them disappear as well.  If you add those things back,
the comments are gone.  The idea is as follows:

@r@
identifier i;
expression e1;
@@

struct i2c_driver i = {
         .driver = {
-        .of_match_table=e1,
+        .of_match_table=of_match_ptr(e1),
        },
};

@@
identifier r.i;
initialiser i1,i2;
@@

struct i2c_driver i = {
          .driver = {
-         i1,i2,
+         i1,i2,
          },
};

@@
identifier r.i;
initialiser i1,i2,i3;
@@

struct i2c_driver i = {
          .driver = {
-         i1,i2,i3,
+         i1,i2,i3,
          },
};

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Markus Elfring]

> It seems that spatch currently just doesn't understand preprocessor
> directives at all,

I find this view only partly appropriate.
There is preprocessor functionality supported by the semantic patch language.

Example:
https://github.com/coccinelle/coccinelle/blob/cad4c0705f9e37f501531e133d3a47bc56ed0ce2/docs/manual/cocci_syntax.tex#L1125


> just treats them as literal strings.

See also:
https://github.com/coccinelle/coccinelle/blob/cad4c0705f9e37f501531e133d3a47bc56ed0ce2/docs/manual/cocci_syntax.tex#L1765

Additional development attention and further resources can considerably
help to improve the discussed situation also for the Coccinelle software,
can't they?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Markus Elfring]

> #ifdefs are comments.

I wonder about this wording.

I hope that the corresponding software situation can be improved further
for complete support of preprocessor functionality.


> If you remove a contiguous sequence of things, the comments between them
> disappear as well.

Such a source code change handling is usually reasonable.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Julia Lawall]

On Thu, 6 Jun 2019, Markus Elfring wrote:

> > It seems that spatch currently just doesn't understand preprocessor
> > directives at all,
>
> I find this view only partly appropriate.
> There is preprocessor functionality supported by the semantic patch language.
>
> Example:
> https://github.com/coccinelle/coccinelle/blob/cad4c0705f9e37f501531e133d3a47bc56ed0ce2/docs/manual/cocci_syntax.tex#L1125
>
>
> > just treats them as literal strings.
>
> See also:
> https://github.com/coccinelle/coccinelle/blob/cad4c0705f9e37f501531e133d3a47bc56ed0ce2/docs/manual/cocci_syntax.tex#L1765

To be less cryptic, Coccinelle supprorts matching and transformation for
#include and #define, and addition of comments pragmas and ifdefs.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Julia Lawall]

On Thu, 6 Jun 2019, Markus Elfring wrote:

> > #ifdefs are comments.
>
> I wonder about this wording.

Ifdefs are parsed as comments.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Markus Elfring]

> Ifdefs are parsed as comments.

I am curious under which circumstances this software situation
can be changed for the better support of preprocessor functionality.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[David Young]

On Thu, Jun 06, 2019 at 04:20:24PM +0200, Markus Elfring wrote:
> > Ifdefs are parsed as comments.
> 
> I am curious under which circumstances this software situation
> can be changed for the better support of preprocessor functionality.

It may not support your particular use case, and it is not nearly
as flexible as `spatch` (in principle) is, but a program called
`unifdef` that ships with *BSD systems will replace #ifdef sections
with their content or else the empty string, depending on the
variables you define on its command line.

Dave

-- 
David Young
dyoung@pobox.com    Urbana, IL    (217) 721-9981
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Enrico Weigelt, metux IT consult]

On 06.06.19 12:34, Julia Lawall wrote:
> 
> 
> On Thu, 6 Jun 2019, Markus Elfring wrote:
> 
>>> #ifdefs are comments.
>>
>> I wonder about this wording.
> 
> Ifdefs are parsed as comments.

Any chance that it gets extended to understand at least such rather
simple cases ?


--mtx

-- 
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Enrico Weigelt, metux IT consult]

On 06.06.19 11:39, Julia Lawall wrote:

>> See also:
>> https://github.com/coccinelle/coccinelle/blob/cad4c0705f9e37f501531e133d3a47bc56ed0ce2/docs/manual/cocci_syntax.tex#L1765
> 
> To be less cryptic, Coccinelle supprorts matching and transformation for
> #include and #define, and addition of comments pragmas and ifdefs.

Could you perhaps give an quick example for a newbie ?


--mtx

-- 
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Julia Lawall]

On Fri, 7 Jun 2019, Enrico Weigelt, metux IT consult wrote:

> On 06.06.19 12:34, Julia Lawall wrote:
> >
> >
> > On Thu, 6 Jun 2019, Markus Elfring wrote:
> >
> >>> #ifdefs are comments.
> >>
> >> I wonder about this wording.
> >
> > Ifdefs are parsed as comments.
>
> Any chance that it gets extended to understand at least such rather
> simple cases ?

Maybe it's possible.  A complexity is that structure field initializers
are considered to be unordered.  Ie if you put

- .a = e1,
- .b = e2,

it will also match a structure declaration in which there is

.b = 3,
.a = 4,

But it would have to detect that the ifdefs and endif are attached to
something specific.

Getting the newlines to work when removing and adding back the
initializers is more likely to work in the short term.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Julia Lawall]

On Fri, 7 Jun 2019, Enrico Weigelt, metux IT consult wrote:

> On 06.06.19 11:39, Julia Lawall wrote:
>
> >> See also:
> >> https://github.com/coccinelle/coccinelle/blob/cad4c0705f9e37f501531e133d3a47bc56ed0ce2/docs/manual/cocci_syntax.tex#L1765
> >
> > To be less cryptic, Coccinelle supprorts matching and transformation for
> > #include and #define, and addition of comments pragmas and ifdefs.
>
> Could you perhaps give an quick example for a newbie ?

You can do:

@@
@@

+ #ifdef FOO
  bar();
+ #endif

but not:

@@
@@

- #ifdef FOO
  bar();
- #endif

You can also do

@@
expression e;
@@

- #define XXX e

or various other things that you may expect to be able to do with #define.

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Markus Elfring]

> but not:
>
> @@
> @@
>
> - #ifdef FOO
>   bar();
> - #endif

The deletion of these preprocessor statements is not directly supported
in the shown way by the Coccinelle software at the moment.
But I got another code transformation idea which would not be so convenient.

The semantic patch language supports the metavariable type “statement list”
for a while.
https://github.com/coccinelle/coccinelle/blob/cad4c0705f9e37f501531e133d3a47bc56ed0ce2/docs/manual/cocci_syntax.tex#L209

I imagine then that the function call can be “intentionally” deleted
and added back in such a simple SmPL script example.

* Should any extra C code vanish after such a special adjustment
  because specific parts were not restored?

* Can it make sense here to change a bit of code even if it was originally
  intended to keep it untouched?


Will similar software updates become more challenging for the initially
described handling of designated initialisers for known data structures?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1257 bytes --]



On Fri, 7 Jun 2019, Markus Elfring wrote:

> > but not:
> >
> > @@
> > @@
> >
> > - #ifdef FOO
> >   bar();
> > - #endif
>
> The deletion of these preprocessor statements is not directly supported
> in the shown way by the Coccinelle software at the moment.
> But I got another code transformation idea which would not be so convenient.
>
> The semantic patch language supports the metavariable type “statement list”
> for a while.
> https://github.com/coccinelle/coccinelle/blob/cad4c0705f9e37f501531e133d3a47bc56ed0ce2/docs/manual/cocci_syntax.tex#L209
>
> I imagine then that the function call can be “intentionally” deleted
> and added back in such a simple SmPL script example.
>
> * Should any extra C code vanish after such a special adjustment
>   because specific parts were not restored?
>
> * Can it make sense here to change a bit of code even if it was originally
>   intended to keep it untouched?
>
>
> Will similar software updates become more challenging for the initially
> described handling of designated initialisers for known data structures?

This is in the spirit of the solution I already proposed.  The original
issue was about structure field initializations, not statements, so
statement list is not appropriate.

julia

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Markus Elfring]

>> Will similar software updates become more challenging for the initially
>> described handling of designated initialisers for known data structures?
>
> This is in the spirit of the solution I already proposed.

How do you think about the following SmPL transformation approach?

@replacement@
constant text;
expression value;
identifier my_name;
type driver_type;
@@
 static driver_type my_name =
 {
 .driver =
 {
-.name = text,
+.name = text,
 .of_match_table =
+                  of_match_ptr(
                                value
+                              )
 ,
 }, ...
 };


Test result:
elfring@Sonne:~/Projekte/Linux/next-patched> spatch ~/Projekte/Coccinelle/Probe/Weigelt5.cocci sound/soc/codecs/pcm3060-i2c.c
…
@@ -43,10 +43,10 @@ MODULE_DEVICE_TABLE(of, pcm3060_of_match
 #endif /* CONFIG_OF */

 static struct i2c_driver pcm3060_i2c_driver = {
+#ifdef CONFIG_OF
 	.driver = {
 		.name = "pcm3060",
-#ifdef CONFIG_OF
-		.of_match_table = pcm3060_of_match,
+		.of_match_table = of_match_ptr(pcm3060_of_match),
 #endif /* CONFIG_OF */
 	},
 	.id_table = pcm3060_i2c_id,


I would find it questionable that the software combination
“Coccinelle 1.0.7-00206-gfdcc6d79 (OCaml 4.07.1)” suggests to move
a conditional preprocessor statement to an other source code place.

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Markus Elfring]

>> Will similar software updates become more challenging for the initially
>> described handling of designated initialisers for known data structures?
>
> This is in the spirit of the solution I already proposed.

Would you like to compare transformation results also for the following SmPL
change specification?

@replacement@
constant text;
expression value;
identifier my_name;
type driver_type;
@@
 static
 driver_type my_name = {
-                      .driver = {
-                                .name = text,
-                                .of_match_table = value
+                      .driver = {
+                                .name = text,
+                                .of_match_table = of_match_ptr(value)
                                 ,
                                 }, ...
                       };


Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Markus Elfring]

> Just wrapping the assignment into the macro call turned out to be easy.

I am curious how your transformation approaches will evolve further.


> But I haven't found a way to remove the now #ifdef :(

The support for preprocessor functionality is limited by the semantic patch
language so far.
Would you like to try another SmPL script variant out?

@replacement@
constant text;
expression value;
identifier my_name;
type driver_type;
@@
 static
 driver_type
 my_name = {
           .driver = {
-                    .name = text,
-                    .of_match_table = value,
-                    },
+                    .name = text,
+                    .of_match_table = of_match_ptr(value),
+                    },
 ...
           };


Do you find the following test result acceptable finally?


elfring@Sonne:~/Projekte/Linux/next-patched> spatch ~/Projekte/Coccinelle/Probe/Weigelt8.cocci sound/soc/codecs/pcm3060-i2c.c
init_defs_builtins: /usr/local/bin/../lib/coccinelle/standard.h
…
@@ -45,9 +45,7 @@ MODULE_DEVICE_TABLE(of, pcm3060_of_match
 static struct i2c_driver pcm3060_i2c_driver = {
 	.driver = {
 		.name = "pcm3060",
-#ifdef CONFIG_OF
-		.of_match_table = pcm3060_of_match,
-#endif /* CONFIG_OF */
+		.of_match_table = of_match_ptr(pcm3060_of_match),
 	},
 	.id_table = pcm3060_i2c_id,
 	.probe = pcm3060_i2c_probe,


Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] How to replace obsolete #ifdef's?

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1716 bytes --]



On Fri, 7 Jun 2019, Markus Elfring wrote:

> > Just wrapping the assignment into the macro call turned out to be easy.
>
> I am curious how your transformation approaches will evolve further.
>
>
> > But I haven't found a way to remove the now #ifdef :(
>
> The support for preprocessor functionality is limited by the semantic patch
> language so far.
> Would you like to try another SmPL script variant out?
>
> @replacement@
> constant text;
> expression value;
> identifier my_name;
> type driver_type;
> @@
>  static
>  driver_type
>  my_name = {
>            .driver = {
> -                    .name = text,
> -                    .of_match_table = value,
> -                    },
> +                    .name = text,
> +                    .of_match_table = of_match_ptr(value),

I guess that there is no guarantee that these are the only fields.

julia

> +                    },
>  ...
>            };
>
>
> Do you find the following test result acceptable finally?
>
>
> elfring@Sonne:~/Projekte/Linux/next-patched> spatch ~/Projekte/Coccinelle/Probe/Weigelt8.cocci sound/soc/codecs/pcm3060-i2c.c
> init_defs_builtins: /usr/local/bin/../lib/coccinelle/standard.h
> …
> @@ -45,9 +45,7 @@ MODULE_DEVICE_TABLE(of, pcm3060_of_match
>  static struct i2c_driver pcm3060_i2c_driver = {
>  	.driver = {
>  		.name = "pcm3060",
> -#ifdef CONFIG_OF
> -		.of_match_table = pcm3060_of_match,
> -#endif /* CONFIG_OF */
> +		.of_match_table = of_match_ptr(pcm3060_of_match),
>  	},
>  	.id_table = pcm3060_i2c_id,
>  	.probe = pcm3060_i2c_probe,
>
>
> Regards,
> Markus
> _______________________________________________
> Cocci mailing list
> Cocci@systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Handling of designated initialisers by SmPL?

[Markus Elfring]

> Would you like to try another SmPL script variant out?

@replacement@
constant text;
expression value;
@@
 .driver = {
-          .name = text,
-          .of_match_table = value,
-          },
+          .name = text,
+          .of_match_table = of_match_ptr(value),
+          },


How do you think about the following response by the Coccinelle software?

elfring@Sonne:~/Projekte/Linux/next-patched> spatch ~/Projekte/Coccinelle/Probe/Weigelt9.cocci sound/soc/codecs/pcm3060-i2c.c
…
minus: parse error:
  File "/home/elfring/Projekte/Coccinelle/Probe/Weigelt9.cocci", line 5, column 1, charpos = 51
  around = '.',
  whole content =  .driver = {


Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci