* [cocci] Does SmPL support regex groups? @ 2022-03-08 23:15 Eric Wheeler 2022-03-09 5:55 ` Julia Lawall ` (2 more replies) 0 siblings, 3 replies; 18+ messages in thread From: Eric Wheeler @ 2022-03-08 23:15 UTC (permalink / raw) To: cocci Hi all, I would like to replace this: void on_struct_save_as_gnuplot_activate( GtkMenuItem *menuitem, gpointer user_data) { file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, "*.gplot", NULL, "untitled.gplot", rc_config.working_dir ); } And extract the extension ".gplot" as an identifier and pass it to the function get_nec_filename_stem, like so: void on_struct_save_as_gnuplot_activate( GtkMenuItem *menuitem, gpointer user_data) { file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, "*.gplot", NULL, get_nec_filename_stem(newfn, ".gplot", PATH_MAX), rc_config.working_dir ); } The SmPL patch below works for what I want statically, but notice the commented `expression E`: Is there a way to regex-group `expression E =~` with a parenthesis and then specify something like E[0] to pick up the grouped regex match? @ replace_untitled @ identifier F; parameter list PL; expression list EL1, EL2; //expression E =~ "\"untitled\.(.*)\""; @@ F(PL) { + char newfn[PATH_MAX]; ... file_chooser = Open_Filechooser(EL1, - "untitled.gplot", + get_nec_filename_stem(newfn, ".gplot", PATH_MAX), EL2); ... } This doesn't work, but it explains what I'm trying to do: @ replace_untitled @ identifier F; parameter list PL; expression list EL1, EL2; expression E =~ "untitled.(.*)"; @@ F(PL) { + char newfn[PATH_MAX]; ... file_chooser = Open_Filechooser(EL1, - E, + get_nec_filename_stem(newfn, E[0], PATH_MAX), EL2); ... } It replaces as follows and does match "untitled.(.*)", but of course [0] is incorrect SmPL grammar: @@ -250,10 +253,13 @@ on_struct_save_as_gnuplot_activate( GtkMenuItem *menuitem, gpointer user_data) { + char newfn[PATH_MAX]; /* Open file chooser to save structure image */ SetFlag( STRUCT_GNUPLOT_SAVE ); file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, - "*.gplot", NULL, "untitled.gplot", rc_config.working_dir ); + "*.gplot", NULL, + get_nec_filename_stem(newfn, "untitled.gplot"[0], PATH_MAX), + rc_config.working_dir ); } I tried something like this: expression F =~ "\..*"; expression E =~ "untitled" ##F; but that is invalid too. Ideas? -- Eric Wheeler ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-08 23:15 [cocci] Does SmPL support regex groups? Eric Wheeler @ 2022-03-09 5:55 ` Julia Lawall 2022-03-09 20:56 ` Eric Wheeler 2022-03-09 18:38 ` Markus Elfring 2022-03-22 14:30 ` Nicolas Palix 2 siblings, 1 reply; 18+ messages in thread From: Julia Lawall @ 2022-03-09 5:55 UTC (permalink / raw) To: Eric Wheeler; +Cc: cocci Regex groups are not supported. You can use python to do whatever you want with the matched terms. See demos/pythontococci.cocci julia On Tue, 8 Mar 2022, Eric Wheeler wrote: > Hi all, > > I would like to replace this: > > void on_struct_save_as_gnuplot_activate( > GtkMenuItem *menuitem, > gpointer user_data) > { > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > "*.gplot", NULL, "untitled.gplot", rc_config.working_dir ); > } > > And extract the extension ".gplot" as an identifier and pass it to the > function get_nec_filename_stem, like so: > > void on_struct_save_as_gnuplot_activate( > GtkMenuItem *menuitem, > gpointer user_data) > { > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > "*.gplot", NULL, get_nec_filename_stem(newfn, ".gplot", PATH_MAX), rc_config.working_dir ); > } > > The SmPL patch below works for what I want statically, but notice the > commented `expression E`: > > Is there a way to regex-group `expression E =~` with a parenthesis and then specify > something like E[0] to pick up the grouped regex match? > > @ replace_untitled @ > identifier F; > parameter list PL; > expression list EL1, EL2; > //expression E =~ "\"untitled\.(.*)\""; > @@ > > F(PL) > { > + char newfn[PATH_MAX]; > ... > file_chooser = Open_Filechooser(EL1, > - "untitled.gplot", > + get_nec_filename_stem(newfn, ".gplot", PATH_MAX), > EL2); > ... > } > > > This doesn't work, but it explains what I'm trying to do: > > @ replace_untitled @ > identifier F; > parameter list PL; > expression list EL1, EL2; > expression E =~ "untitled.(.*)"; > @@ > > F(PL) > { > + char newfn[PATH_MAX]; > ... > file_chooser = Open_Filechooser(EL1, > - E, > + get_nec_filename_stem(newfn, E[0], PATH_MAX), > EL2); > ... > } > > It replaces as follows and does match "untitled.(.*)", but of course [0] is incorrect SmPL grammar: > > @@ -250,10 +253,13 @@ on_struct_save_as_gnuplot_activate( > GtkMenuItem *menuitem, > gpointer user_data) > { > + char newfn[PATH_MAX]; > /* Open file chooser to save structure image */ > SetFlag( STRUCT_GNUPLOT_SAVE ); > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > - "*.gplot", NULL, "untitled.gplot", rc_config.working_dir ); > + "*.gplot", NULL, > + get_nec_filename_stem(newfn, "untitled.gplot"[0], PATH_MAX), > + rc_config.working_dir ); > } > > I tried something like this: > expression F =~ "\..*"; > expression E =~ "untitled" ##F; > but that is invalid too. > > Ideas? > > > -- > Eric Wheeler > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-09 5:55 ` Julia Lawall @ 2022-03-09 20:56 ` Eric Wheeler 2022-03-09 21:20 ` Julia Lawall 0 siblings, 1 reply; 18+ messages in thread From: Eric Wheeler @ 2022-03-09 20:56 UTC (permalink / raw) To: Julia Lawall; +Cc: cocci > Regex groups are not supported. You can use python to do whatever you > want with the matched terms. See demos/pythontococci.cocci Thanks Julia. It seems the Coccinelle =~ matches do not support escaping. I was getting errors trying to match like so: expression E =~ "\.gplot"; It works without the \ but errors with the . File "cocci/untitled-fix.cocci", line 5, column 35, charpos = 144 around = '\.', whole content = expression EXT, arg, E =~ "untitled\.(.*)"; It failed scaping \" as well but now I'm unable to reproduce the problem. Sounds like python is the way to go, built-in regex's are useful and might need a bit of cleanup. +1 for group and escaping support if it might be a trivial patch. -- Eric Wheeler On Wed, 9 Mar 2022, Julia Lawall wrote: > > julia > > On Tue, 8 Mar 2022, Eric Wheeler wrote: > > > Hi all, > > > > I would like to replace this: > > > > void on_struct_save_as_gnuplot_activate( > > GtkMenuItem *menuitem, > > gpointer user_data) > > { > > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > > "*.gplot", NULL, "untitled.gplot", rc_config.working_dir ); > > } > > > > And extract the extension ".gplot" as an identifier and pass it to the > > function get_nec_filename_stem, like so: > > > > void on_struct_save_as_gnuplot_activate( > > GtkMenuItem *menuitem, > > gpointer user_data) > > { > > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > > "*.gplot", NULL, get_nec_filename_stem(newfn, ".gplot", PATH_MAX), rc_config.working_dir ); > > } > > > > The SmPL patch below works for what I want statically, but notice the > > commented `expression E`: > > > > Is there a way to regex-group `expression E =~` with a parenthesis and then specify > > something like E[0] to pick up the grouped regex match? > > > > @ replace_untitled @ > > identifier F; > > parameter list PL; > > expression list EL1, EL2; > > //expression E =~ "\"untitled\.(.*)\""; > > @@ > > > > F(PL) > > { > > + char newfn[PATH_MAX]; > > ... > > file_chooser = Open_Filechooser(EL1, > > - "untitled.gplot", > > + get_nec_filename_stem(newfn, ".gplot", PATH_MAX), > > EL2); > > ... > > } > > > > > > This doesn't work, but it explains what I'm trying to do: > > > > @ replace_untitled @ > > identifier F; > > parameter list PL; > > expression list EL1, EL2; > > expression E =~ "untitled.(.*)"; > > @@ > > > > F(PL) > > { > > + char newfn[PATH_MAX]; > > ... > > file_chooser = Open_Filechooser(EL1, > > - E, > > + get_nec_filename_stem(newfn, E[0], PATH_MAX), > > EL2); > > ... > > } > > > > It replaces as follows and does match "untitled.(.*)", but of course [0] is incorrect SmPL grammar: > > > > @@ -250,10 +253,13 @@ on_struct_save_as_gnuplot_activate( > > GtkMenuItem *menuitem, > > gpointer user_data) > > { > > + char newfn[PATH_MAX]; > > /* Open file chooser to save structure image */ > > SetFlag( STRUCT_GNUPLOT_SAVE ); > > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > > - "*.gplot", NULL, "untitled.gplot", rc_config.working_dir ); > > + "*.gplot", NULL, > > + get_nec_filename_stem(newfn, "untitled.gplot"[0], PATH_MAX), > > + rc_config.working_dir ); > > } > > > > I tried something like this: > > expression F =~ "\..*"; > > expression E =~ "untitled" ##F; > > but that is invalid too. > > > > Ideas? > > > > > > -- > > Eric Wheeler > > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-09 20:56 ` Eric Wheeler @ 2022-03-09 21:20 ` Julia Lawall 2022-03-10 2:25 ` Eric Wheeler 2022-03-10 8:39 ` Markus Elfring 0 siblings, 2 replies; 18+ messages in thread From: Julia Lawall @ 2022-03-09 21:20 UTC (permalink / raw) To: Eric Wheeler; +Cc: cocci On Wed, 9 Mar 2022, Eric Wheeler wrote: > > Regex groups are not supported. You can use python to do whatever you > > want with the matched terms. See demos/pythontococci.cocci > > Thanks Julia. It seems the Coccinelle =~ matches do not support escaping. > I was getting errors trying to match like so: > > expression E =~ "\.gplot"; > > It works without the \ but errors with the . > File "cocci/untitled-fix.cocci", line 5, column 35, charpos = 144 > around = '\.', > whole content = expression EXT, arg, E =~ "untitled\.(.*)"; > > It failed scaping \" as well but now I'm unable to reproduce the problem. > Sounds like python is the way to go, built-in regex's are useful and might > need a bit of cleanup. I would imagine that you need to put "\\.gplot" I would imagine that escaping is trivial to addres. I have no idea what one would even want from grouping. That is, what notation do you expect to use to refer to the groups. And which group, if there are multiple metavariables that use regular expressions? julia > > +1 for group and escaping support if it might be a trivial patch. > > > -- > Eric Wheeler > > On Wed, 9 Mar 2022, Julia Lawall wrote: > > > > > > julia > > > > On Tue, 8 Mar 2022, Eric Wheeler wrote: > > > > > Hi all, > > > > > > I would like to replace this: > > > > > > void on_struct_save_as_gnuplot_activate( > > > GtkMenuItem *menuitem, > > > gpointer user_data) > > > { > > > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > > > "*.gplot", NULL, "untitled.gplot", rc_config.working_dir ); > > > } > > > > > > And extract the extension ".gplot" as an identifier and pass it to the > > > function get_nec_filename_stem, like so: > > > > > > void on_struct_save_as_gnuplot_activate( > > > GtkMenuItem *menuitem, > > > gpointer user_data) > > > { > > > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > > > "*.gplot", NULL, get_nec_filename_stem(newfn, ".gplot", PATH_MAX), rc_config.working_dir ); > > > } > > > > > > The SmPL patch below works for what I want statically, but notice the > > > commented `expression E`: > > > > > > Is there a way to regex-group `expression E =~` with a parenthesis and then specify > > > something like E[0] to pick up the grouped regex match? > > > > > > @ replace_untitled @ > > > identifier F; > > > parameter list PL; > > > expression list EL1, EL2; > > > //expression E =~ "\"untitled\.(.*)\""; > > > @@ > > > > > > F(PL) > > > { > > > + char newfn[PATH_MAX]; > > > ... > > > file_chooser = Open_Filechooser(EL1, > > > - "untitled.gplot", > > > + get_nec_filename_stem(newfn, ".gplot", PATH_MAX), > > > EL2); > > > ... > > > } > > > > > > > > > This doesn't work, but it explains what I'm trying to do: > > > > > > @ replace_untitled @ > > > identifier F; > > > parameter list PL; > > > expression list EL1, EL2; > > > expression E =~ "untitled.(.*)"; > > > @@ > > > > > > F(PL) > > > { > > > + char newfn[PATH_MAX]; > > > ... > > > file_chooser = Open_Filechooser(EL1, > > > - E, > > > + get_nec_filename_stem(newfn, E[0], PATH_MAX), > > > EL2); > > > ... > > > } > > > > > > It replaces as follows and does match "untitled.(.*)", but of course [0] is incorrect SmPL grammar: > > > > > > @@ -250,10 +253,13 @@ on_struct_save_as_gnuplot_activate( > > > GtkMenuItem *menuitem, > > > gpointer user_data) > > > { > > > + char newfn[PATH_MAX]; > > > /* Open file chooser to save structure image */ > > > SetFlag( STRUCT_GNUPLOT_SAVE ); > > > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > > > - "*.gplot", NULL, "untitled.gplot", rc_config.working_dir ); > > > + "*.gplot", NULL, > > > + get_nec_filename_stem(newfn, "untitled.gplot"[0], PATH_MAX), > > > + rc_config.working_dir ); > > > } > > > > > > I tried something like this: > > > expression F =~ "\..*"; > > > expression E =~ "untitled" ##F; > > > but that is invalid too. > > > > > > Ideas? > > > > > > > > > -- > > > Eric Wheeler > > > > > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-09 21:20 ` Julia Lawall @ 2022-03-10 2:25 ` Eric Wheeler 2022-03-10 6:12 ` Julia Lawall 2022-03-10 9:16 ` Markus Elfring 2022-03-10 8:39 ` Markus Elfring 1 sibling, 2 replies; 18+ messages in thread From: Eric Wheeler @ 2022-03-10 2:25 UTC (permalink / raw) To: Julia Lawall; +Cc: cocci On Wed, 9 Mar 2022, Julia Lawall wrote: > On Wed, 9 Mar 2022, Eric Wheeler wrote: > > > Regex groups are not supported. You can use python to do whatever you > > > want with the matched terms. See demos/pythontococci.cocci > > > > Thanks Julia. It seems the Coccinelle =~ matches do not support escaping. > > I was getting errors trying to match like so: > > > > expression E =~ "\.gplot"; > > > > It works without the \ but errors with the . > > File "cocci/untitled-fix.cocci", line 5, column 35, charpos = 144 > > around = '\.', > > whole content = expression EXT, arg, E =~ "untitled\.(.*)"; > > > > It failed scaping \" as well but now I'm unable to reproduce the problem. > > Sounds like python is the way to go, built-in regex's are useful and might > > need a bit of cleanup. > > I would imagine that you need to put "\\.gplot" > I would imagine that escaping is trivial to addres. Running this against a few examples: @ r @ constant C =~ "(a)\\.([a-z])"; @@ -C +"foo" No match: main() { char *a = "a.b"; } No match: main() { char *a = "aXb"; } No match: main() { char *a = "a\\.b"; } Does match: main() { char *a = "a\\b"; } ... so I'm not sure what escapes are doing with the "." in the regex, it seems to have disappeared! It occurs to me that to match an actual "." I could have used "[.]" in the regex instead of trying to escape with "\." However, for quotes, this spatch: @ r @ constant C =~ "\\\"ab\\\""; @@ -C +"foo" Does indeed match this: main() { char *a = "\"ab\""; } and it makes sense that \ needs escaped and the " needs escaped, thus \\\" matches the C string of "\"". It might be more intuative to the SmPL author if the regex matched without all those extra escapes. > I have no idea what one would even want from grouping. See my example below for a real-world case, but first, a simpler contrived example... > That is, what notation do you expect to use to refer to the groups. > And which group, if there are multiple metavariables that use regular > expressions? I would think that the group would be bound to the metavariable since the metavariable is tied 1:1 to the regex. I'm not sure what syntax fits best with the Coccinelle paradigm, so whatever works best for you in terms of group reference. In the examples below I use @n where n is the group number. Here is a possible example to replace the string "az" with "z" and "run quick" with "quick run" where @n could be the group number: @ r @ constant C =~ "(a)([a-z])"; constant D =~ "(run|walk) (quick)"; @@ -C +C@2 ... -D +D@2 D@1 /* "quick run", or possibly, "quick walk" */ In my example from the original thread post (below), I was trying to replace all constant strings "untitled(\..*$)" with f(group1) because the refactor changes things like "untitled.png" to only pass the extension ".png" (get_nec_filename_stem() will use lookup the original filename so "untitled" needs to be dropped and the extension placed in the function call.). There were lots of "untitled.png", "untitled.nec", "untitled.s1p", etc., so a quick regex could pull out the filename extension in this case and put it in the function call in the right position: @ replace_untitled @ identifier F; parameter list PL; expression list EL1, EL2; constant C =~ "untitled(\..*)"; @@ F(PL) { + char newfn[PATH_MAX]; ... file_chooser = Open_Filechooser(EL1, - C, + get_nec_filename_stem(newfn, C@1, PATH_MAX), EL2); ... } I'm not sure how common of a use-case this was, but if OCaml already supports regex groups then perhaps it is trivial to create a "group" notation in SmPL for metavariables using regex's. (Or perhaps not, I really haven't used OCaml so I can't comment on how easy that would be!) Anyway, my needs for this are already met with a combination of SmPL cleverness and a bit of manual intervention, so no need to rush about trying to implement this on my behalf unless it would benefit others. -Eric ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-10 2:25 ` Eric Wheeler @ 2022-03-10 6:12 ` Julia Lawall 2022-03-10 9:16 ` Markus Elfring 1 sibling, 0 replies; 18+ messages in thread From: Julia Lawall @ 2022-03-10 6:12 UTC (permalink / raw) To: Eric Wheeler; +Cc: cocci > I'm not sure how common of a use-case this was, but if OCaml already > supports regex groups then perhaps it is trivial to create a "group" > notation in SmPL for metavariables using regex's. (Or perhaps not, I > really haven't used OCaml so I can't comment on how easy that would be!) The problem is how to refer to the groups in the semantic patch. I guess that the @n notation is possible, but I'm not enthusiastic. But on the ihter hand I never use regular expressions either. There is still the option of script code, as in demos/pythontococci. julia ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-10 2:25 ` Eric Wheeler 2022-03-10 6:12 ` Julia Lawall @ 2022-03-10 9:16 ` Markus Elfring 2022-03-10 22:03 ` Eric Wheeler 1 sibling, 1 reply; 18+ messages in thread From: Markus Elfring @ 2022-03-10 9:16 UTC (permalink / raw) To: Eric Wheeler; +Cc: Julia Lawall, cocci > Anyway, my needs for this are already met with a combination of SmPL > cleverness and a bit of manual intervention, Would you like to share a bit more information about a (temporary) solution which you find good enough at the moment? > so no need to rush about trying to implement this on my behalf Thanks for your nice feedback. > unless it would benefit others. I guess that further demonstrations for the application of (advanced) regular expressions in combination with the semantic patch language can help to clarify corresponding use cases. Regards, Markus ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-10 9:16 ` Markus Elfring @ 2022-03-10 22:03 ` Eric Wheeler 2022-03-11 11:12 ` Nicolas Palix 0 siblings, 1 reply; 18+ messages in thread From: Eric Wheeler @ 2022-03-10 22:03 UTC (permalink / raw) To: Markus Elfring; +Cc: Julia Lawall, cocci On Thu, 10 Mar 2022, Markus Elfring wrote: > > Anyway, my needs for this are already met with a combination of SmPL > > cleverness and a bit of manual intervention, > > Would you like to share a bit more information about a (temporary) solution > which you find good enough at the moment? This was the resulting diff from git after doing the complete refactor to remove "untitled" from default filenames: @@ -991,6 +1019,7 @@ on_freqplots_save_as_activate( GtkMenuItem *menuitem, gpointer user_data) { + char newfn[PATH_MAX]; saveas_drawingarea = freqplots_drawingarea; saveas_width = freqplots_width; saveas_height = freqplots_height; @@ -998,7 +1027,8 @@ on_freqplots_save_as_activate( /* Open file chooser to save frequency plots */ SetFlag( IMAGE_SAVE ); file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, - "*.png", NULL, _("untitled.png"), rc_config.working_dir ); + "*.png", NULL, get_nec_filename_stem(newfn, ".png", PATH_MAX), + rc_config.working_dir ); } This was the Coccinelle SmPL: @ replace_untitled @ identifier F != {on_nec2_save_as_clicked}; parameter list PL; expression list EL1, EL2; expression EXT, arg, E =~ "untitled\.(.*)"; @@ F(PL) { + char newfn[PATH_MAX]; ... file_chooser = Open_Filechooser(GTK_FILE_CHOOSER_ACTION_SAVE, EXT, arg, - _(E), + get_nec_filename_stem(newfn, EXT, PATH_MAX), EL2); ... } but that just moved the existing "*.png" in the metavariable EXT into EXT within get_nec_filename_stem(), which wasn't quite enough, because get_nec_filename_stem() wants ".png" not "*.png": so I manually deleted the *'s. It seems that I use Coccinelle daily: there is always something that is easier solved with SmPL than manual editing, even if there is some minor intervention after SmPL does the heavy lifting. -- Eric Wheeler > > > > so no need to rush about trying to implement this on my behalf > > Thanks for your nice feedback. > > > > unless it would benefit others. > > I guess that further demonstrations for the application of (advanced) > regular expressions in combination with the semantic patch language can help > to clarify corresponding use cases. > > Regards, > Markus > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-10 22:03 ` Eric Wheeler @ 2022-03-11 11:12 ` Nicolas Palix 2022-03-11 12:39 ` Julia Lawall 2022-03-12 7:00 ` Markus Elfring 0 siblings, 2 replies; 18+ messages in thread From: Nicolas Palix @ 2022-03-11 11:12 UTC (permalink / raw) To: Eric Wheeler; +Cc: Julia Lawall, cocci Hi, If there is not too much extensions, maybe you can enumerate them in the SmPL, or generate a SmPL script per-extension after a collecting phase. The Regexp are the ones from the Ocaml Str module, or PCRE according to the configuration. https://github.com/coccinelle/coccinelle/blob/master/globals/regexp_pcre.ml So, the group definition are legal for the matching, but maybe depends of the underlying Regexp used. Moreover, but because we have Python and OCaml escapes with more power to generate thing, using group seems hard to use from a SmPL syntax perspective. I like the C@1 notation to get the group, but how would you say to concatenate it with a prefix/suffix either static or variable (from another metavariable) ? As your example suggests, maybe keeping only the first group if any is defined in the + part would be great, and all the bind string otherwise. I retrieve the matching phase here https://github.com/coccinelle/coccinelle/blob/master/engine/cocci_vs_c.ml Julia, where can I find the + part that where the bind string is used ? Can we still have the Regexp at that point ? On 10/03/2022 23:03, Eric Wheeler wrote: > On Thu, 10 Mar 2022, Markus Elfring wrote: >>> Anyway, my needs for this are already met with a combination of SmPL >>> cleverness and a bit of manual intervention, >> >> Would you like to share a bit more information about a (temporary) solution >> which you find good enough at the moment? > > This was the resulting diff from git after doing the complete refactor to > remove "untitled" from default filenames: > > @@ -991,6 +1019,7 @@ on_freqplots_save_as_activate( > GtkMenuItem *menuitem, > gpointer user_data) > { > + char newfn[PATH_MAX]; > saveas_drawingarea = freqplots_drawingarea; > saveas_width = freqplots_width; > saveas_height = freqplots_height; > @@ -998,7 +1027,8 @@ on_freqplots_save_as_activate( > /* Open file chooser to save frequency plots */ > SetFlag( IMAGE_SAVE ); > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > - "*.png", NULL, _("untitled.png"), rc_config.working_dir ); > + "*.png", NULL, get_nec_filename_stem(newfn, ".png", PATH_MAX), > + rc_config.working_dir ); > } > > This was the Coccinelle SmPL: > > @ replace_untitled @ > identifier F != {on_nec2_save_as_clicked}; > parameter list PL; > expression list EL1, EL2; > expression EXT, arg, E =~ "untitled\.(.*)"; > @@ > > F(PL) > { > + char newfn[PATH_MAX]; > ... > file_chooser = Open_Filechooser(GTK_FILE_CHOOSER_ACTION_SAVE, EXT, arg, > - _(E), > + get_nec_filename_stem(newfn, EXT, PATH_MAX), > EL2); > ... > } > > but that just moved the existing "*.png" in the metavariable EXT into EXT > within get_nec_filename_stem(), which wasn't quite enough, because > get_nec_filename_stem() wants ".png" not "*.png": so I manually deleted > the *'s. > > It seems that I use Coccinelle daily: there is always something that is > easier solved with SmPL than manual editing, even if there is some minor > intervention after SmPL does the heavy lifting. > > > -- > Eric Wheeler > > > >> >> >>> so no need to rush about trying to implement this on my behalf >> >> Thanks for your nice feedback. >> >> >>> unless it would benefit others. >> >> I guess that further demonstrations for the application of (advanced) >> regular expressions in combination with the semantic patch language can help >> to clarify corresponding use cases. >> >> Regards, >> Markus >> -- Nicolas Palix ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-11 11:12 ` Nicolas Palix @ 2022-03-11 12:39 ` Julia Lawall 2022-03-12 7:00 ` Markus Elfring 1 sibling, 0 replies; 18+ messages in thread From: Julia Lawall @ 2022-03-11 12:39 UTC (permalink / raw) To: Nicolas Palix; +Cc: Eric Wheeler, cocci On Fri, 11 Mar 2022, Nicolas Palix wrote: > Hi, > > If there is not too much extensions, maybe you can enumerate them in the SmPL, > or generate a SmPL script per-extension after a collecting phase. > > > The Regexp are the ones from the Ocaml Str module, or PCRE according to the > configuration. > https://github.com/coccinelle/coccinelle/blob/master/globals/regexp_pcre.ml > > So, the group definition are legal for the matching, but maybe depends of the > underlying Regexp used. > > Moreover, but because we have Python and OCaml escapes with more power to > generate thing, using group seems hard to use from a SmPL syntax perspective. > > I like the C@1 notation to get the group, but how would you say > to concatenate it with a prefix/suffix either static or variable (from another > metavariable) ? > > As your example suggests, maybe keeping only the first group if any is defined > in the + part would be great, and all the bind string otherwise. > > I retrieve the matching phase here > https://github.com/coccinelle/coccinelle/blob/master/engine/cocci_vs_c.ml > > Julia, where can I find the + part that where the bind string > is used ? Can we still have the Regexp at that point ? The actual use of the + code is in parsing_c/unparse_cocci.ml. For the regexp to be available, it would have to be stored in the matavars_binding type that is defined in parsing_c/ast_c.ml References to grouping only makes sense in + code. References to grouping would be ok for metavariables of type constant. But they could lead to problems for other kinds of metavariables. For example, a expression metavariable could end up bound to something that is not an expression. So Coccinelle would have to reparse the extracted string like one does in python or ocaml scripts (pythontococci.ml or ocamltococci.ml). julia > > > On 10/03/2022 23:03, Eric Wheeler wrote: > > On Thu, 10 Mar 2022, Markus Elfring wrote: > > > > Anyway, my needs for this are already met with a combination of SmPL > > > > cleverness and a bit of manual intervention, > > > > > > Would you like to share a bit more information about a (temporary) > > > solution > > > which you find good enough at the moment? > > > > This was the resulting diff from git after doing the complete refactor to > > remove "untitled" from default filenames: > > > > @@ -991,6 +1019,7 @@ on_freqplots_save_as_activate( > > GtkMenuItem *menuitem, > > gpointer user_data) > > { > > + char newfn[PATH_MAX]; > > saveas_drawingarea = freqplots_drawingarea; > > saveas_width = freqplots_width; > > saveas_height = freqplots_height; > > @@ -998,7 +1027,8 @@ on_freqplots_save_as_activate( > > /* Open file chooser to save frequency plots */ > > SetFlag( IMAGE_SAVE ); > > file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, > > - "*.png", NULL, _("untitled.png"), rc_config.working_dir ); > > + "*.png", NULL, get_nec_filename_stem(newfn, ".png", PATH_MAX), > > + rc_config.working_dir ); > > } > > > > This was the Coccinelle SmPL: > > > > @ replace_untitled @ > > identifier F != {on_nec2_save_as_clicked}; > > parameter list PL; > > expression list EL1, EL2; > > expression EXT, arg, E =~ "untitled\.(.*)"; > > @@ > > > > F(PL) > > { > > + char newfn[PATH_MAX]; > > ... > > file_chooser = Open_Filechooser(GTK_FILE_CHOOSER_ACTION_SAVE, > > EXT, arg, > > - _(E), > > + get_nec_filename_stem(newfn, EXT, PATH_MAX), > > EL2); > > ... > > } > > > > but that just moved the existing "*.png" in the metavariable EXT into EXT > > within get_nec_filename_stem(), which wasn't quite enough, because > > get_nec_filename_stem() wants ".png" not "*.png": so I manually deleted > > the *'s. > > > > It seems that I use Coccinelle daily: there is always something that is > > easier solved with SmPL than manual editing, even if there is some minor > > intervention after SmPL does the heavy lifting. > > > > > > -- > > Eric Wheeler > > > > > > > > > > > > > > > > so no need to rush about trying to implement this on my behalf > > > > > > Thanks for your nice feedback. > > > > > > > > > > unless it would benefit others. > > > > > > I guess that further demonstrations for the application of (advanced) > > > regular expressions in combination with the semantic patch language can > > > help > > > to clarify corresponding use cases. > > > > > > Regards, > > > Markus > > > > > > -- > Nicolas Palix > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-11 11:12 ` Nicolas Palix 2022-03-11 12:39 ` Julia Lawall @ 2022-03-12 7:00 ` Markus Elfring 1 sibling, 0 replies; 18+ messages in thread From: Markus Elfring @ 2022-03-12 7:00 UTC (permalink / raw) To: Nicolas Palix, cocci; +Cc: Eric Wheeler, Julia Lawall > I like the C@1 notation to get the group, I suggest to reconsider this specification approach further. > but how would you say to concatenate it with a prefix/suffix either static or variable > (from another metavariable) ? I would like to point other software design options out. The syntax for the @ operator of the semantic patch language is described in the way that it should be connected with variable identifiers. https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5479fc964d209209e84ddd5caf3f395667721e7c/docs/manual/cocci_syntax.tex#L410 https://github.com/coccinelle/coccinelle/blob/ae337fce1512ff15aabc3ad5b6d2e537f97ab62a/docs/manual/cocci_syntax.tex#L410 One of the supported metavariables has got the type “position”. This metavariable kind has got the special property that it refers to a concrete source code place (and not to a syntax element). Thus I propose the addition of support for the metavariable type “capture group”. See also: 2015-08-27: Feature request “Reuse of content from capturing groups in regular expressions” https://github.com/coccinelle/coccinelle/issues/46 I propose also to take the support for named capture groups into account. https://www.regular-expressions.info/named.html Thus I imagine that there is a need to enable the specification of an index for special metavariable types. The corresponding index can be specified as a number or an identifier. I guess that path specifications can become relevant for references to captured contents from nested groups. Regards, Markus ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-09 21:20 ` Julia Lawall 2022-03-10 2:25 ` Eric Wheeler @ 2022-03-10 8:39 ` Markus Elfring 1 sibling, 0 replies; 18+ messages in thread From: Markus Elfring @ 2022-03-10 8:39 UTC (permalink / raw) To: Julia Lawall; +Cc: Eric Wheeler, cocci > I have no idea what one would even want from grouping. I find this feedback surprising. > That is, what notation do you expect to use to refer to the groups. > And which group, if there are multiple metavariables that use regular expressions? I see possibilities like the following. A) The specification of these regular expressions is connected with a constraint for specific metavariables. Such metavariables could provide additional data by special access operators and corresponding programming interfaces. B) If direct access to special information would not be offered, I imagine that additional options could be specified for the evaluation of capture groups (and the selection of regex engines). Additional variables could store captured contents. Regards, Markus ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-08 23:15 [cocci] Does SmPL support regex groups? Eric Wheeler 2022-03-09 5:55 ` Julia Lawall @ 2022-03-09 18:38 ` Markus Elfring 2022-03-09 20:51 ` Eric Wheeler 2022-03-22 14:30 ` Nicolas Palix 2 siblings, 1 reply; 18+ messages in thread From: Markus Elfring @ 2022-03-09 18:38 UTC (permalink / raw) To: Eric Wheeler; +Cc: cocci > And extract the extension ".gplot" as an identifier It seems that you are looking for a string literal (which would contain a file name). > and pass it to the function get_nec_filename_stem, like so: Literals can be found with the means of the semantic patch language by the metavariable type “constant”. https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5479fc964d209209e84ddd5caf3f395667721e7c/docs/manual/cocci_syntax.tex#L216 https://github.com/coccinelle/coccinelle/blob/ae337fce1512ff15aabc3ad5b6d2e537f97ab62a/docs/manual/cocci_syntax.tex#L216 > The SmPL patch below works for what I want statically, but notice the > commented `expression E`: > > Is there a way to regex-group `expression E =~` with a parenthesis and then specify > something like E[0] to pick up the grouped regex match? Would you like to achieve any further software evolution also according to my feature request “Reuse of content from capturing groups in regular expressions” from 2015-08-27? https://github.com/coccinelle/coccinelle/issues/46 > @ replace_untitled @ > identifier F; > parameter list PL; > expression list EL1, EL2; > //expression E =~ "\"untitled\.(.*)\""; > @@ > > F(PL) > { > + char newfn[PATH_MAX]; > ... > file_chooser = Open_Filechooser(EL1, > - "untitled.gplot", > + get_nec_filename_stem(newfn, ".gplot", PATH_MAX), > EL2); > ... > } I would like to point the possibility out that SmPL ellipses can be used instead of the metavariables “PL”, “EL1” and “EL2”. > Ideas? Would you get any further ideas from the clarification approach for the topic “Development challenges around using the function “make_expr” for expression construction by SmPL” (from 2016-10-27)? https://github.com/coccinelle/coccinelle/issues/88 Excerpt from my script: @initialize:python@ @@ import re @find_update_candidate@ constant input_string; expression input_context; position pos; @@ seq_printf@pos(input_context, input_string); @script:python selection@ call; param << find_update_candidate.input_string; pos << find_update_candidate.pos; text; @@ match = re.match('"([^"]+)"', param) if match: … Regards, Markus ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-09 18:38 ` Markus Elfring @ 2022-03-09 20:51 ` Eric Wheeler 2022-03-09 21:16 ` Markus Elfring 0 siblings, 1 reply; 18+ messages in thread From: Eric Wheeler @ 2022-03-09 20:51 UTC (permalink / raw) To: Markus Elfring; +Cc: cocci [-- Attachment #1: Type: text/plain, Size: 2523 bytes --] On Wed, 9 Mar 2022, Markus Elfring wrote: > > And extract the extension ".gplot" as an identifier > > > It seems that you are looking for a string literal > (which would contain a file name). > > > > > and pass it to the function get_nec_filename_stem, like so: > > Literals can be found with the means of the semantic patch language > by the metavariable type “constant”. > > https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5479fc964d209209e84ddd5caf3f395667721e7c/docs/manual/cocci_syntax.tex#L216 > https://github.com/coccinelle/coccinelle/blob/ae337fce1512ff15aabc3ad5b6d2e537f97ab62a/docs/manual/cocci_syntax.tex#L216 Can literals match substrings like ".gplot" for cut-and-splice? > > The SmPL patch below works for what I want statically, but notice the > > commented `expression E`: > > > > Is there a way to regex-group `expression E =~` with a parenthesis and then specify > > something like E[0] to pick up the grouped regex match? > > > Would you like to achieve any further software evolution also according to > my feature request “Reuse of content from capturing groups in regular expressions” > from 2015-08-27? > https://github.com/coccinelle/coccinelle/issues/46 Github says 404 not found. -Eric > > > > > @ replace_untitled @ > > identifier F; > > parameter list PL; > > expression list EL1, EL2; > > //expression E =~ "\"untitled\.(.*)\""; > > @@ > > > > F(PL) > > { > > + char newfn[PATH_MAX]; > > ... > > file_chooser = Open_Filechooser(EL1, > > - "untitled.gplot", > > + get_nec_filename_stem(newfn, ".gplot", PATH_MAX), > > EL2); > > ... > > } > > > I would like to point the possibility out that SmPL ellipses > can be used instead of the metavariables “PL”, “EL1” and “EL2”. > > > > > Ideas? > > > Would you get any further ideas from the clarification approach for the topic > “Development challenges around using the function “make_expr” for expression > construction by SmPL” (from 2016-10-27)? > https://github.com/coccinelle/coccinelle/issues/88 > > Excerpt from my script: > > @initialize:python@ > @@ > import re > > @find_update_candidate@ > constant input_string; > expression input_context; > position pos; > @@ > seq_printf@pos(input_context, input_string); > > @script:python selection@ > call; > param << find_update_candidate.input_string; > pos << find_update_candidate.pos; > text; > @@ > match = re.match('"([^"]+)"', param) > if match: > … > > > Regards, > Markus > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-09 20:51 ` Eric Wheeler @ 2022-03-09 21:16 ` Markus Elfring 0 siblings, 0 replies; 18+ messages in thread From: Markus Elfring @ 2022-03-09 21:16 UTC (permalink / raw) To: Eric Wheeler; +Cc: cocci >> Literals can be found with the means of the semantic patch language >> by the metavariable type “constant”. >> >> https://gitlab.inria.fr/coccinelle/coccinelle/-/blob/5479fc964d209209e84ddd5caf3f395667721e7c/docs/manual/cocci_syntax.tex#L216 >> https://github.com/coccinelle/coccinelle/blob/ae337fce1512ff15aabc3ad5b6d2e537f97ab62a/docs/manual/cocci_syntax.tex#L216 > Can literals match substrings like ".gplot" for cut-and-splice? The Coccinelle software contains some known limitations. Some limits can be adjusted if you can manage to activate extra possibilities by switching parts of the corresponding data processing to one of the supported scripting languages (like Python and OCaml). The application of (advanced) regular expressions provides various chances, doesn't it? Regards, Markus ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-08 23:15 [cocci] Does SmPL support regex groups? Eric Wheeler 2022-03-09 5:55 ` Julia Lawall 2022-03-09 18:38 ` Markus Elfring @ 2022-03-22 14:30 ` Nicolas Palix 2022-03-22 19:26 ` Markus Elfring 2022-03-23 6:00 ` Eric Wheeler 2 siblings, 2 replies; 18+ messages in thread From: Nicolas Palix @ 2022-03-22 14:30 UTC (permalink / raw) To: Eric Wheeler, cocci For the record, is it the expected SmPL ? @ fu @ identifier F,file_chooser; parameter list PL; expression list EL1, EL2; constant U =~ "untitled"; @@ F(PL) { ... file_chooser = Open_Filechooser(EL1, U, EL2); ... } @script:python py@ U << fu.U; C; @@ print U # Rewrite U to extract the extension coccinelle.C = cocci.make_expr("\".gplot\"") @ replace_untitled @ identifier F,file_chooser; parameter list PL; expression list EL1, EL2; constant fu.U; expression py.C; @@ F(PL) { + char newfn[PATH_MAX]; ... file_chooser = Open_Filechooser(EL1, - U, + get_nec_filename_stem(newfn, C, PATH_MAX), EL2); ... } -- Nicolas Palix ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-22 14:30 ` Nicolas Palix @ 2022-03-22 19:26 ` Markus Elfring 2022-03-23 6:00 ` Eric Wheeler 1 sibling, 0 replies; 18+ messages in thread From: Markus Elfring @ 2022-03-22 19:26 UTC (permalink / raw) To: Nicolas Palix; +Cc: Eric Wheeler, cocci > For the record, is it the expected SmPL ? Your SmPL script variant points specific development ideas out. > @script:python py@ > U << fu.U; > C; > @@ > > print U # Rewrite U to extract the extension > coccinelle.C = cocci.make_expr("\".gplot\"") I find that this code part would need another case distinction (after the use of a capturing group within a regular expression) so that the file extension will be extracted as expected for the desired source code transformation. Regards, Markus ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [cocci] Does SmPL support regex groups? 2022-03-22 14:30 ` Nicolas Palix 2022-03-22 19:26 ` Markus Elfring @ 2022-03-23 6:00 ` Eric Wheeler 1 sibling, 0 replies; 18+ messages in thread From: Eric Wheeler @ 2022-03-23 6:00 UTC (permalink / raw) To: Nicolas Palix; +Cc: cocci On Tue, 22 Mar 2022, Nicolas Palix wrote: > For the record, is it the expected SmPL ? > > @ fu @ > identifier F,file_chooser; > parameter list PL; > expression list EL1, EL2; > constant U =~ "untitled"; > @@ > > F(PL) > { > ... > file_chooser = Open_Filechooser(EL1, U, EL2); > ... > } > > @script:python py@ > U << fu.U; > C; > @@ > > print U # Rewrite U to extract the extension > coccinelle.C = cocci.make_expr("\".gplot\"") The extension varies, there were half a dozen or so: I had to replace several Open_Filechooser() calls that had different extensions. Here is the diff for that commit: https://github.com/KJ7LNW/xnec2c/commit/9fb0381c24365311e3bb1fc8158f18112864ebfb#diff-1d0efcfbefd013aedf0477bd522a1e4506339c5ef239a6ef770f39270747c14cL222 Here is one such diff for .png: file_chooser = Open_Filechooser( GTK_FILE_CHOOSER_ACTION_SAVE, - "*.png", NULL, "untitled.png", rc_config.working_dir ); + "*.png", NULL, get_nec_filename_stem(newfn, ".png", PATH_MAX), rc_config.working_dir ); I'm not so familiar with Python, so forgive my likely incorrect syntax, but this might work: coccinelle.C = re.match("(\..*)$", U).group(1) -Eric > @ replace_untitled @ > identifier F,file_chooser; > parameter list PL; > expression list EL1, EL2; > constant fu.U; > expression py.C; > @@ > > F(PL) > { > + char newfn[PATH_MAX]; > ... > file_chooser = Open_Filechooser(EL1, > - U, > + get_nec_filename_stem(newfn, C, PATH_MAX), > EL2); > ... > } > > > > -- > Nicolas Palix > ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2022-03-23 6:01 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-03-08 23:15 [cocci] Does SmPL support regex groups? Eric Wheeler 2022-03-09 5:55 ` Julia Lawall 2022-03-09 20:56 ` Eric Wheeler 2022-03-09 21:20 ` Julia Lawall 2022-03-10 2:25 ` Eric Wheeler 2022-03-10 6:12 ` Julia Lawall 2022-03-10 9:16 ` Markus Elfring 2022-03-10 22:03 ` Eric Wheeler 2022-03-11 11:12 ` Nicolas Palix 2022-03-11 12:39 ` Julia Lawall 2022-03-12 7:00 ` Markus Elfring 2022-03-10 8:39 ` Markus Elfring 2022-03-09 18:38 ` Markus Elfring 2022-03-09 20:51 ` Eric Wheeler 2022-03-09 21:16 ` Markus Elfring 2022-03-22 14:30 ` Nicolas Palix 2022-03-22 19:26 ` Markus Elfring 2022-03-23 6:00 ` Eric Wheeler
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.