Re: [Cocci] Exit with non-zero status

Re: [Cocci] Exit with non-zero status

[Markus Elfring]

> I am trying to exit spatch with a non-zero exit status when required
> changes have been found.

I got used to the interpretation that a zero exit status
should indicate a successful execution of the program.


>   @finalize:python@
>   @@
>   exit(cnt)
>
> where cnt is incremented by another Python rule everytime a required
> change is found.

I suggest to handle the desired search result counter
by an other data processing interface.
How would you like to treat program execution failures otherwise?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Exit with non-zero status

[Paul Chaignon]

On Wed, Apr 29, 2020 at 08:40:10PM +0200, Markus Elfring wrote:
> > I am trying to exit spatch with a non-zero exit status when required
> > changes have been found.
> 
> I got used to the interpretation that a zero exit status
> should indicate a successful execution of the program.

I agree, but I guess what successful execution means is a matter of
perspective. For example, grep returns a non-zero exit status when it
doesn't match anything. Yet, I generally consider that a successful
execution when I use grep.

> 
> 
> >   @finalize:python@
> >   @@
> >   exit(cnt)
> >
> > where cnt is incremented by another Python rule everytime a required
> > change is found.
> 
> I suggest to handle the desired search result counter
> by an other data processing interface.
> How would you like to treat program execution failures otherwise?

I'm trying to run SmPL programs as part of a CI pipeline, so execution
failure and (buggy) code pattern not found really mean the same in
the end: a human needs to take a look :-)

I understand that is not a common use case and I'm fine using a
workaround. Just wanted to check I wasn't missing something obvious.

Cheers,
Paul

> 
> Regards,
> Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Exit with non-zero status

[Markus Elfring]

>> I suggest to handle the desired search result counter
>> by an other data processing interface.
>> How would you like to treat program execution failures otherwise?
>
> I'm trying to run SmPL programs as part of a CI pipeline, so execution
> failure and (buggy) code pattern not found really mean the same in
> the end: a human needs to take a look :-)

I imagine that such a look can occasionally made a bit more convenient.


> I understand that is not a common use case

Can such a software application become more popular?


> and I'm fine using a workaround.

We have got different views around preferable solutions.
How many information would you like to put into an exit code so far?


> Just wanted to check I wasn't missing something obvious.

Further possibilities should probably be clarified further,
shouldn't they?

Will the usage of additional data structures become more interesting?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Exit with non-zero status

[Paul Chaignon]

On Wed, Apr 29, 2020 at 09:50:21PM +0200, Markus Elfring wrote:
> >> I suggest to handle the desired search result counter
> >> by an other data processing interface.
> >> How would you like to treat program execution failures otherwise?
> >
> > I'm trying to run SmPL programs as part of a CI pipeline, so execution
> > failure and (buggy) code pattern not found really mean the same in
> > the end: a human needs to take a look :-)
> 
> I imagine that such a look can occasionally made a bit more convenient.
> 
> 
> > I understand that is not a common use case
> 
> Can such a software application become more popular?

As far as I understand, if it's not more popular to use Coccinelle in
CIs right now it's because of (1) false positives for some rules, and
(2) the little reliance of the Linux community on CIs.

We rely heavily on CI and probably wouldn't use Coccinelle in the long
term if we couldn't enable it in CI. Without CI, developers and
reviewers would likely forget to execute semantic patches.

So CI use seems to become more popular (even in low-level projects), but
it's not like this exit status issue is a blocker to use Coccinelle in
CI. For the record, I enabled Coccinelle in our CI (GitHub Actions)
yesterday and worked around that issue with the following script:

  make -C bpf/ coccicheck | tee /tmp/stdout
  exit $(grep -c "[prefix of warning messages]" /tmp/stdout)

> 
> 
> > and I'm fine using a workaround.
> 
> We have got different views around preferable solutions.
> How many information would you like to put into an exit code so far?

Please see above script. I currently exit with number of failures, but
having just 0 (success) and 1 (fail) statuses would be enough.

> 
> 
> > Just wanted to check I wasn't missing something obvious.
> 
> Further possibilities should probably be clarified further,
> shouldn't they?
> 
> Will the usage of additional data structures become more interesting?

Not sure I understand the question. What data structures?

Cheers,
Paul
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Exit with non-zero status

[Markus Elfring]

> For the record, I enabled Coccinelle in our CI (GitHub Actions)
> yesterday and worked around that issue with the following script:
>
>   make -C bpf/ coccicheck | tee /tmp/stdout
>   exit $(grep -c "[prefix of warning messages]" /tmp/stdout)

I propose to improve the data processing approach according to your recent
commit “coccinelle: Run coccicheck with GitHub Actions”.
https://github.com/cilium/cilium/commit/35d299a4ee996dc2f4beec3c88b77dedbd00b2ba


> Please see above script. I currently exit with number of failures,
> but having just 0 (success) and 1 (fail) statuses would be enough.

I suggest to stick to a more traditional handling of the program
exit status because of development concerns around software portability.


>> Will the usage of additional data structures become more interesting?
>
> Not sure I understand the question. What data structures?

Examples:

* Ordinary text files for the standard output and error channels.

* Would you like to store any search results into database tables?

* How do you think about to perform data transmissions by higher level
  programming interfaces for distributed processes?


Which software extensions can become more interesting also for
applications around your continuous integration system?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Exit with non-zero status

[Paul Chaignon]

On Tue, May 05, 2020 at 01:08:26PM +0200, Markus Elfring wrote:
> > For the record, I enabled Coccinelle in our CI (GitHub Actions)
> > yesterday and worked around that issue with the following script:
> >
> >   make -C bpf/ coccicheck | tee /tmp/stdout
> >   exit $(grep -c "[prefix of warning messages]" /tmp/stdout)
> 
> I propose to improve the data processing approach according to your recent
> commit “coccinelle: Run coccicheck with GitHub Actions”.
> https://github.com/cilium/cilium/commit/35d299a4ee996dc2f4beec3c88b77dedbd00b2ba
> 
> 
> > Please see above script. I currently exit with number of failures,
> > but having just 0 (success) and 1 (fail) statuses would be enough.
> 
> I suggest to stick to a more traditional handling of the program
> exit status because of development concerns around software portability.
> 
> 
> >> Will the usage of additional data structures become more interesting?
> >
> > Not sure I understand the question. What data structures?
> 
> Examples:
> 
> * Ordinary text files for the standard output and error channels.
> 
> * Would you like to store any search results into database tables?
> 
> * How do you think about to perform data transmissions by higher level
>   programming interfaces for distributed processes?
> 
> 
> Which software extensions can become more interesting also for
> applications around your continuous integration system?

I see. We're not planning to use any of the above in the medium term.
We're only planning to add Coccinelle rules in the future. If we need more
complex post-processing, we'll likely implement it in GitHub Actions and
not directly in Coccinelle scripts.

Paul
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Exit with non-zero status

[Markus Elfring]

> We're only planning to add Coccinelle rules in the future.

I am curious how software extensions will evolve further in this area.


> If we need more complex post-processing,

I imagine that the selection of nicer data structures can occasionally
become more interesting.


> we'll likely implement it in GitHub Actions and

Do you feel more comfortable with this interface at the moment?


> not directly in Coccinelle scripts.

Can they help any more also for your data processing needs?

Regards,
Markus
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Exit with non-zero status

[Julia Lawall]

On Wed, 29 Apr 2020, Paul Chaignon wrote:

> On Wed, Apr 29, 2020 at 05:07:56PM +0200, Julia Lawall wrote:
> >
> >
> > On Wed, 29 Apr 2020, Paul Chaignon wrote:
> >
> > > Hi everyone,
> > >
> > > I am trying to exit spatch with a non-zero exit status when required
> > > changes have been found. I have tried to add a rule with finalize:python
> > > and the following code:
> > >
> > >   @finalize:python@
> > >   @@
> > >   exit(cnt)
> > >
> > > where cnt is incremented by another Python rule everytime a required
> > > change is found. This works as expected except for the --in-place option
> > > which becomes ineffective.
> > >
> > > Is this a bug, or is finalize:python not intended to be used in that
> > > manner? Is it possible to overwrite spatch's exit status?
> >
> > I'm actually surprised that this would ever work...
> >
> > Finalize was intended for processing information in general, eg printing
> > some statistics about what was matched.
> >
> > Would it be feasible to print cnt instead?
>
> Sure. I can easily get around this issue by e.g. grepping for spatch's
> messages in a wrapper script. Getting spatch to return a non-zero exit
> status is just a convenience. I thought that would be a fairly common
> use case.

I'll add it to the "list of features someone has asked for and might get
implemented some day in the future"...

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Exit with non-zero status

[Paul Chaignon]

On Wed, Apr 29, 2020 at 05:07:56PM +0200, Julia Lawall wrote:
> 
> 
> On Wed, 29 Apr 2020, Paul Chaignon wrote:
> 
> > Hi everyone,
> >
> > I am trying to exit spatch with a non-zero exit status when required
> > changes have been found. I have tried to add a rule with finalize:python
> > and the following code:
> >
> >   @finalize:python@
> >   @@
> >   exit(cnt)
> >
> > where cnt is incremented by another Python rule everytime a required
> > change is found. This works as expected except for the --in-place option
> > which becomes ineffective.
> >
> > Is this a bug, or is finalize:python not intended to be used in that
> > manner? Is it possible to overwrite spatch's exit status?
> 
> I'm actually surprised that this would ever work...
> 
> Finalize was intended for processing information in general, eg printing
> some statistics about what was matched.
> 
> Would it be feasible to print cnt instead?

Sure. I can easily get around this issue by e.g. grepping for spatch's
messages in a wrapper script. Getting spatch to return a non-zero exit
status is just a convenience. I thought that would be a fairly common
use case.

Thanks for the quick answer as always :)
Paul

> 
> julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Exit with non-zero status

[Julia Lawall]

On Wed, 29 Apr 2020, Paul Chaignon wrote:

> Hi everyone,
>
> I am trying to exit spatch with a non-zero exit status when required
> changes have been found. I have tried to add a rule with finalize:python
> and the following code:
>
>   @finalize:python@
>   @@
>   exit(cnt)
>
> where cnt is incremented by another Python rule everytime a required
> change is found. This works as expected except for the --in-place option
> which becomes ineffective.
>
> Is this a bug, or is finalize:python not intended to be used in that
> manner? Is it possible to overwrite spatch's exit status?

I'm actually surprised that this would ever work...

Finalize was intended for processing information in general, eg printing
some statistics about what was matched.

Would it be feasible to print cnt instead?

julia
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

[Cocci] Exit with non-zero status

[Paul Chaignon]

Hi everyone,

I am trying to exit spatch with a non-zero exit status when required
changes have been found. I have tried to add a rule with finalize:python
and the following code:

  @finalize:python@
  @@
  exit(cnt)

where cnt is incremented by another Python rule everytime a required
change is found. This works as expected except for the --in-place option
which becomes ineffective.

Is this a bug, or is finalize:python not intended to be used in that
manner? Is it possible to overwrite spatch's exit status?

Thanks,
Paul
_______________________________________________
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci