Re: [cocci] Optimizing *.cocci rules by concat'ing them

[Julia Lawall <julia.lawall@inria.fr>]

[-- Attachment #1: Type: text/plain, Size: 2764 bytes --]



On Thu, 1 Sep 2022, Ævar Arnfjörð Bjarmason wrote:

> Are there reasons for why $subject is a generally bad idea, i.e. given a
> set of *.cocci rules in a directory instead of something like:
>
> 	for c in cocci/*.cocci
> 	do
> 		spatch --sp-file $c --patch . *.c
> 	done
>
> Doing:
>
> 	cat cocci/*.cocci >ALL.cocci
> 	spatch --sp-file ALL.cocci --patch . *.c
>
> I'm aware that rules in general may rely on previous transformations,
> such that the 3rd rule in a file might not fire unless the other 2 have
> previously changed the code.

I think that the main problem is the opposite of what you suggest.  Two
.cocci files might define rules that have the same names.

I don't see any other problems offhand.  If one rule makes a change that
somehow affects a later semantic patch, then that should be ok, assuming
that the change made by the first rule was actually a fix.

Another issue is that you can't mix rules that make changes with rules
that just search (*).  You need at least some command line options, like
-D patch in make coccicheck, to keep them apart.

julia

>
> But aside from any such undeclared dependencies, which surely I'll have
> trouble with either way. I.e. I'd get to the 2nd rule in A.cocci, apply
> that to my sources, and only then on a second run would the 1st rule in
> B.cocci fire, but it should have been in A.cocci to begin with, and
> depending on the 2nd rule in that file.
>
> Are there reasons for why this is a Bad Idea?
>
> The motivation here (and this is probably burying the lede) is that I
> implemented a "spatchcache", which can cache previous runs of "spatch"
> in a local Redis (or with a custom command, if you don't have Redis, I
> started out with a dumb FS-based cache).
>
> It uses the approach of piggy-backing on GCC and Clang's ability to dump
> a full list of all the files that a given *.c file depends on.
>
> This is usually used to drive better incremental "make" support, but I'm
> (ab)using it to implement an "spatchcache". If the hash of a the target
> file *and* everything it depends on *and* spatch's path, flags etc. is
> the same I can cache the output of "spatch".
>
> See [1] for a series to implement this for git.git. I'd previously mused
> on this list in [2] about this approach (but hadn't finished it up until
> now).
>
> But even then git's fairly slow in running "make coccicheck", and of
> course it does nothing to speed up an initial run. So concat'ing the
> *.cocci files we use seems like a viable option. But I thought I'd ask
> first in case I'm running into somewell-known caveat.
>
> Thanks!
>
> 1. https://lore.kernel.org/git/cover-v2-0.9-00000000000-20220831T205130Z-avarab@gmail.com/
> 2. https://lore.kernel.org/cocci/211116.86lf1op65q.gmgdl@evledraar.gmail.com/
>