[Cocci] Matching against a declarer macro

[Cocci] Matching against a declarer macro

[Christoph Böhmwalder]

Hi again,

I'm having a little trouble matching against this line of code:

RB_DECLARE_CALLBACKS_MAX(static, augment_callbacks, struct 
drbd_interval, rb, sector_t, end, NODE_END);

This is especially tricky because it contains a lot of macro magic.
I think the biggest problem is the first argument, which is the keyword 
"static". What do I use to match against this? expression? identifier? 
symbol?

Also, the "augment_callbacks" is not really an identifier either, it 
just gets used to generate the function names. But what is it? An 
expression?

@@
typedef sector_t;
declarer name RB_DECLARE_CALLBACKS_MAX;

identifier augment_callbacks;
identifier rb;
identifier end;
identifier NODE_END;
@@
-RB_DECLARE_CALLBACKS_MAX(static, augment_callbacks, struct 
drbd_interval, rb, sector_t, end, NODE_END);

Nothing I have tried has made it match yet.

Any ideas on how to solve this would be appreciated, thanks!

--
Christoph Böhmwalder
LINBIT | Keeping the Digital World Running
DRBD HA —  Disaster Recovery — Software defined Storage
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Matching against a declarer macro

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1706 bytes --]



On Fri, 27 Mar 2020, Christoph Böhmwalder wrote:

> Hi again,
>
> I'm having a little trouble matching against this line of code:
>
> RB_DECLARE_CALLBACKS_MAX(static, augment_callbacks, struct
> drbd_interval, rb, sector_t, end, NODE_END);
>
> This is especially tricky because it contains a lot of macro magic.
> I think the biggest problem is the first argument, which is the keyword
> "static". What do I use to match against this? expression? identifier?
> symbol?
>
> Also, the "augment_callbacks" is not really an identifier either, it
> just gets used to generate the function names. But what is it? An
> expression?
>
> @@
> typedef sector_t;
> declarer name RB_DECLARE_CALLBACKS_MAX;
>
> identifier augment_callbacks;
> identifier rb;
> identifier end;
> identifier NODE_END;
> @@
> -RB_DECLARE_CALLBACKS_MAX(static, augment_callbacks, struct
> drbd_interval, rb, sector_t, end, NODE_END);
>
> Nothing I have tried has made it match yet.
>
> Any ideas on how to solve this would be appreciated, thanks!

Are you sure that the C code is parsed successfully?  I'm not at all sure
that static is allowed in an argument list.  Types are allowed, bu static
is only part of a type.

For augment_callbacks, either identifier or expression would be fine.
Coccinelle has no idea what is going to happen to augment_callbacks
afterwards.  It just sees a sequence of characters and classifies it as an
identifier.

julia

>
> --
> Christoph Böhmwalder
> LINBIT | Keeping the Digital World Running
> DRBD HA —  Disaster Recovery — Software defined Storage
> _______________________________________________
> Cocci mailing list
> Cocci@systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Matching against a declarer macro

[Christoph Böhmwalder]

Am 27.03.20 um 16:47 schrieb Julia Lawall:> Are you sure that the C code 
is parsed successfully?  I'm not at all sure
> that static is allowed in an argument list.  Types are allowed, bu static
> is only part of a type.

I'm pretty sure it is parsed successfully. At least spatch doesn't 
complain about it, no matter how many debug flags I specify.

I really only want to swap out the last parameter, but I'm having 
trouble coming up with the syntax. This should match, right?

@@
declarer name RB_DECLARE_CALLBACKS_MAX;
identifier NODE_END;
@@
RB_DECLARE_CALLBACKS_MAX(...,
- NODE_END
+ compute_subtree_last
  );

> For augment_callbacks, either identifier or expression would be fine.
> Coccinelle has no idea what is going to happen to augment_callbacks
> afterwards.  It just sees a sequence of characters and classifies it as an
> identifier.

That's what I thought, thank you for confirming.

> julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Matching against a declarer macro

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 1201 bytes --]



On Fri, 27 Mar 2020, Christoph Böhmwalder wrote:

> Am 27.03.20 um 16:47 schrieb Julia Lawall:> Are you sure that the C code is
> parsed successfully?  I'm not at all sure
> > that static is allowed in an argument list.  Types are allowed, bu static
> > is only part of a type.
>
> I'm pretty sure it is parsed successfully. At least spatch doesn't complain
> about it, no matter how many debug flags I specify.

Did you try spatch --parse-c file.c?

>
> I really only want to swap out the last parameter, but I'm having trouble
> coming up with the syntax. This should match, right?
>
> @@
> declarer name RB_DECLARE_CALLBACKS_MAX;
> identifier NODE_END;
> @@
> RB_DECLARE_CALLBACKS_MAX(...,
> - NODE_END
> + compute_subtree_last
>  );

If the code is getting parsed, this should be fine.

Do you want to actually match NODE_END?  If so, it shouldn't be declared
as a metavariable.

julia

> > For augment_callbacks, either identifier or expression would be fine.
> > Coccinelle has no idea what is going to happen to augment_callbacks
> > afterwards.  It just sees a sequence of characters and classifies it as an
> > identifier.
>
> That's what I thought, thank you for confirming.
>
> > julia
>

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Matching against a declarer macro

[Christoph Böhmwalder]

Am 27.03.20 um 17:15 schrieb Julia Lawall:
> 
> 
> On Fri, 27 Mar 2020, Christoph Böhmwalder wrote:
> 
>> Am 27.03.20 um 16:47 schrieb Julia Lawall:> Are you sure that the C code is
>> parsed successfully?  I'm not at all sure
>>> that static is allowed in an argument list.  Types are allowed, bu static
>>> is only part of a type.
>>
>> I'm pretty sure it is parsed successfully. At least spatch doesn't complain
>> about it, no matter how many debug flags I specify.
> 
> Did you try spatch --parse-c file.c?

It seems like RB_DECLARE_CALLBACKS_MAX is recognized as a "known macro"? 
Looks like it is able to parse this bit of code just fine:

$ spatch --parse-c drbd/drbd_interval.c --debug
init_defs_builtins: /usr/local/bin/../lib/coccinelle/standard.h

PARSING: drbd/drbd_interval.c
parse error
  = error in drbd/drbd_interval.h; set verbose_parsing for more info
badcount: 5
bad:                                    unsigned int);
bad:
bad: #define drbd_for_each_overlap(i, root, sector, size)               \
bad:    for (i = drbd_find_overlap(root, sector, size);         \
bad:         i;                                                 \
BAD:!!!!!            i = drbd_next_overlap(i, sector, size))
(ONCE) CPP-MACRO: found known macro = RB_DECLARE_CALLBACKS_MAX
passed:static , augment_callbacks , struct drbd_interval , rb ,
passed:sector_t , end , NODE_END
-----------------------------------------------------------------------
maybe 10 most problematic tokens
-----------------------------------------------------------------------
-----------------------------------------------------------------------
NB total files = 1; perfect = 1; pbs = 0; timeout = 0; =========> 100%
nb good = 160,  nb passed = 2 =========> 1.23% passed
nb good = 160,  nb bad = 0 =========> 100.00% good or passed

> Do you want to actually match NODE_END?  If so, it shouldn't be declared
> as a metavariable.

Right, I do want to exactly match NODE_END, so I removed the 
declaration; still no match though. If it is detected as a known macro 
it should be able to be substituted, right?

> 
> julia
>

Thanks,
--
Christoph Böhmwalder
LINBIT | Keeping the Digital World Running
DRBD HA —  Disaster Recovery — Software defined Storage
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Matching against a declarer macro

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 2638 bytes --]



On Sun, 29 Mar 2020, Christoph Böhmwalder wrote:

> Am 27.03.20 um 17:15 schrieb Julia Lawall:
> >
> >
> > On Fri, 27 Mar 2020, Christoph Böhmwalder wrote:
> >
> > > Am 27.03.20 um 16:47 schrieb Julia Lawall:> Are you sure that the C code
> > > is
> > > parsed successfully?  I'm not at all sure
> > > > that static is allowed in an argument list.  Types are allowed, bu
> > > > static
> > > > is only part of a type.
> > >
> > > I'm pretty sure it is parsed successfully. At least spatch doesn't
> > > complain
> > > about it, no matter how many debug flags I specify.
> >
> > Did you try spatch --parse-c file.c?
>
> It seems like RB_DECLARE_CALLBACKS_MAX is recognized as a "known macro"? Looks
> like it is able to parse this bit of code just fine:
>
> $ spatch --parse-c drbd/drbd_interval.c --debug
> init_defs_builtins: /usr/local/bin/../lib/coccinelle/standard.h
>
> PARSING: drbd/drbd_interval.c
> parse error
>  = error in drbd/drbd_interval.h; set verbose_parsing for more info
> badcount: 5
> bad:                                    unsigned int);
> bad:
> bad: #define drbd_for_each_overlap(i, root, sector, size)               \
> bad:    for (i = drbd_find_overlap(root, sector, size);         \
> bad:         i;                                                 \
> BAD:!!!!!            i = drbd_next_overlap(i, sector, size))
> (ONCE) CPP-MACRO: found known macro = RB_DECLARE_CALLBACKS_MAX
> passed:static , augment_callbacks , struct drbd_interval , rb ,
> passed:sector_t , end , NODE_END

It's not parsing anything.  You can see that here.  It is passing, ie
ignoring, the entire command line.

I don't remember the exact situation with this code.  Is there no
semicolon at the end of the macro line?

julia


> -----------------------------------------------------------------------
> maybe 10 most problematic tokens
> -----------------------------------------------------------------------
> -----------------------------------------------------------------------
> NB total files = 1; perfect = 1; pbs = 0; timeout = 0; =========> 100%
> nb good = 160,  nb passed = 2 =========> 1.23% passed
> nb good = 160,  nb bad = 0 =========> 100.00% good or passed
>
> > Do you want to actually match NODE_END?  If so, it shouldn't be declared
> > as a metavariable.
>
> Right, I do want to exactly match NODE_END, so I removed the declaration;
> still no match though. If it is detected as a known macro it should be able to
> be substituted, right?
>
> >
> > julia
> >
>
> Thanks,
> --
> Christoph Böhmwalder
> LINBIT | Keeping the Digital World Running
> DRBD HA —  Disaster Recovery — Software defined Storage
>

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Matching against a declarer macro

[Christoph Böhmwalder]

Am 29.03.20 um 11:20 schrieb Julia Lawall>
> It's not parsing anything.  You can see that here.  It is passing, ie
> ignoring, the entire command line.

Right, I misinterpreted that "passed" as "these parameters were passed
to the macro:".

> 
> I don't remember the exact situation with this code.  Is there no
> semicolon at the end of the macro line?

This is the exact piece of code:

RB_DECLARE_CALLBACKS_MAX(static, augment_callbacks, struct 
drbd_interval, rb, sector_t, end, NODE_END);

Putting this (and only this) in a test.c file produces the same problem:

$ spatch --parse-c test.c --debug --verbose-parsing
init_defs_builtins: /usr/local/bin/../lib/coccinelle/standard.h

PARSING: test.c
Warning: PARSING: type defaults to 'int'; value = [((((0, 2)))); (Tag1 
("", (("", 0, 0, 0, ""), -1)), (0), ((0, 0, 0, 0)), 0, (3))]
ERROR-RECOV: end of file while in recovery mode
parsing pass2: try again
TYPEDEF:_handle_typedef=false. Not normal if don't come from exn
Warning: PARSING: type defaults to 'int'; value = [((((0, 2)))); (Tag1 
("", (("", 0, 0, 0, ""), -1)), (0), ((0, 0, 0, 0)), 0, (3))]
ERROR-RECOV: end of file while in recovery mode
parsing pass3: try again
(ONCE) CPP-MACRO: found known macro = RB_DECLARE_CALLBACKS_MAX
TYPEDEF:_handle_typedef=false. Not normal if don't come from exn
Warning: PARSING: type defaults to 'int'; value = [[[0]]; ((((0, 2)))); 
(Tag1 ("", (("", 0, 0, 0, ""), -1)), (0), ((0, 0, 0, 0)), 0, (3))]
passed:static , augment_callbacks , struct drbd_interval , rb ,
passed:sector_t , end , NODE_END
-----------------------------------------------------------------------
maybe 10 most problematic tokens
-----------------------------------------------------------------------
-----------------------------------------------------------------------
NB total files = 1; perfect = 1; pbs = 0; timeout = 0; =========> 100%
nb good = 1,  nb passed = 2 =========> 66.67% passed
nb good = 1,  nb bad = 0 =========> 100.00% good or passed


Does this mean that this is just "unparseable"?

> 
> julia
> 
> 
>> -----------------------------------------------------------------------
>> maybe 10 most problematic tokens
>> -----------------------------------------------------------------------
>> -----------------------------------------------------------------------
>> NB total files = 1; perfect = 1; pbs = 0; timeout = 0; =========> 100%
>> nb good = 160,  nb passed = 2 =========> 1.23% passed
>> nb good = 160,  nb bad = 0 =========> 100.00% good or passed
>>
>>> Do you want to actually match NODE_END?  If so, it shouldn't be declared
>>> as a metavariable.
>>
>> Right, I do want to exactly match NODE_END, so I removed the declaration;
>> still no match though. If it is detected as a known macro it should be able to
>> be substituted, right?
>>
>>>
>>> julia
>>>
>>
>> Thanks,
>> --
>> Christoph Böhmwalder
>> LINBIT | Keeping the Digital World Running
>> DRBD HA —  Disaster Recovery — Software defined Storage
>>
> 
--
Christoph Böhmwalder
LINBIT | Keeping the Digital World Running
DRBD HA —  Disaster Recovery — Software defined Storage
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Matching against a declarer macro

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 2431 bytes --]



On Sun, 29 Mar 2020, Christoph Böhmwalder wrote:

> Am 29.03.20 um 11:20 schrieb Julia Lawall>
> > It's not parsing anything.  You can see that here.  It is passing, ie
> > ignoring, the entire command line.
>
> Right, I misinterpreted that "passed" as "these parameters were passed
> to the macro:".
>
> >
> > I don't remember the exact situation with this code.  Is there no
> > semicolon at the end of the macro line?
>
> This is the exact piece of code:
>
> RB_DECLARE_CALLBACKS_MAX(static, augment_callbacks, struct drbd_interval, rb,
> sector_t, end, NODE_END);
>
> Putting this (and only this) in a test.c file produces the same problem:
>
> $ spatch --parse-c test.c --debug --verbose-parsing
> init_defs_builtins: /usr/local/bin/../lib/coccinelle/standard.h
>
> PARSING: test.c
> Warning: PARSING: type defaults to 'int'; value = [((((0, 2)))); (Tag1 ("",
> (("", 0, 0, 0, ""), -1)), (0), ((0, 0, 0, 0)), 0, (3))]
> ERROR-RECOV: end of file while in recovery mode
> parsing pass2: try again
> TYPEDEF:_handle_typedef=false. Not normal if don't come from exn
> Warning: PARSING: type defaults to 'int'; value = [((((0, 2)))); (Tag1 ("",
> (("", 0, 0, 0, ""), -1)), (0), ((0, 0, 0, 0)), 0, (3))]
> ERROR-RECOV: end of file while in recovery mode
> parsing pass3: try again
> (ONCE) CPP-MACRO: found known macro = RB_DECLARE_CALLBACKS_MAX
> TYPEDEF:_handle_typedef=false. Not normal if don't come from exn
> Warning: PARSING: type defaults to 'int'; value = [[[0]]; ((((0, 2)))); (Tag1
> ("", (("", 0, 0, 0, ""), -1)), (0), ((0, 0, 0, 0)), 0, (3))]
> passed:static , augment_callbacks , struct drbd_interval , rb ,
> passed:sector_t , end , NODE_END
> -----------------------------------------------------------------------
> maybe 10 most problematic tokens
> -----------------------------------------------------------------------
> -----------------------------------------------------------------------
> NB total files = 1; perfect = 1; pbs = 0; timeout = 0; =========> 100%
> nb good = 1,  nb passed = 2 =========> 66.67% passed
> nb good = 1,  nb bad = 0 =========> 100.00% good or passed
>
>
> Does this mean that this is just "unparseable"?

Everything would be fine if the "static" were not there.

Maybe you could use sed to replace (static by (STATIC in your code base,
then run your script, and then use sed to go the other way...

I don't know if it could be possible to allow static in argument lists.

julia

[-- Attachment #2: Type: text/plain, Size: 136 bytes --]

_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Matching against a declarer macro

[Christoph Böhmwalder]

Am 29.03.20 um 13:24 schrieb Julia Lawall:
> Everything would be fine if the "static" were not there.
> 
> Maybe you could use sed to replace (static by (STATIC in your code base,
> then run your script, and then use sed to go the other way...

The problem is that the scripts I'm writing here are called 
automatically as part of the build process, it's not just a one-shot 
replacement.

FWIW, this can not be parsed either:

RB_DECLARE_CALLBACKS_MAX(STATIC, augment_callbacks, struct 
drbd_interval, rb, sector_t, end, NODE_END);

But this can:

RB_DECLARE_CALLBACKS_MAX(_STATIC, augment_callbacks, struct 
drbd_interval, rb, sector_t, end, NODE_END);


Maybe it would be possible to "#define _STATIC static" for this case -- 
spatch handles that just fine.

> I don't know if it could be possible to allow static in argument lists.

That would be amazing if it's easily possible. If not, I guess we can 
probably just live with the workaround.

> 
> julia
> 
Thanks yet again for all your help.

--
Christoph Böhmwalder
LINBIT | Keeping the Digital World Running
DRBD HA —  Disaster Recovery — Software defined Storage
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci