[cocci] spatch --jobs N missing matches?

* [cocci] spatch --jobs N missing matches?
@ 2022-09-26 18:52 Kees Cook
  2022-09-26 21:14 ` Julia Lawall
  0 siblings, 1 reply; 7+ messages in thread
From: Kees Cook @ 2022-09-26 18:52 UTC (permalink / raw)
  To: cocci; +Cc: Julia Lawall, linux-hardening, Gustavo A. R. Silva

Hi,

I have been unable to figure out what is going wrong with spatch doing
what seems like a simple match, but not finding correct results. Here is
the .cocci file:

----
// "level1" matches a struct ending in a flexible array.
@level1@
identifier inner, flex;
type T;
@@
        struct inner {
                ...
                T flex[];
        };

// "level2" matches a composite flexible array struct (struct ending with "level1")
@level2@
identifier level1.inner;
identifier outer, compflex;
@@
        struct outer {
                ...
                struct inner compflex;
        };

// match memcpy() which has a composite flexible array struct as the destination
@memcpy_compflex_dest depends on level2@
identifier level2.outer, level2.compflex;
struct outer *PTR;
expression SRC, SIZE;
@@

  memcpy(
*       &PTR->compflex
  , SRC, SIZE)

----

I am using spatch on Ubuntu 22.04.1 LTS:

$ spatch --version
spatch version 1.1.1 compiled with OCaml version 4.13.1

But I've also tried this with the latest from git with (worse?) results
(see below).

I'm using the same "include" options as generated by current Linux builds:

$ INCLUDES="-I ./arch/x86/include -I ./arch/x86/include/generated -I ./include -I ./arch/x86/include/uapi -I ./arch/x86/include/generated/uapi -I ./include/uapi -I ./include/generated/uapi --include ./include/linux/compiler-version.h --include ./include/linux/kconfig.h"

And here are the other common arguments:

$ ARGS="--very-quiet $INCLUDES --cocci-file compflex-simple.cocci"

I'm running this against linux-next (20220923).

The first issue I encountered, was that the default didn't match for a
known case:

$ time spatch $ARGS net/sched/cls_u32.c | grep ^---

real    0m0.096s
...

Also it didn't work with "--all-includes":

$ time spatch $ARGS --all-includes net/sched/cls_u32.c | grep ^---

real    0m1.150s
...

In reading the documentation carefully, it seem the desired option is
actually "--recursive-includes" option, which _does_ work:

$ time spatch $ARGS --recursive-includes net/sched/cls_u32.c | grep ^---
--- net/sched/cls_u32.c

real    0m25.332s
...

This takes _much_ longer to run, though. (25 seconds vs 1 ...)

However, if I run this in parallel (using the options shown in the
kernel's build), it does _not_ find the hit in net/sched/cls_u32.c (?!)

$ time spatch $ARGS --recursive-includes --jobs 36 --chunksize 1 --dir . | grep ^---
6594 files match
EXN: Sys_error("./sound/firewire/fireworks/packets-buffer.h: No such file or directory") in ./sound/firewire/fireworks/fireworks_command.c
EXN: Sys_error("./sound/firewire/bebob/lib.h: No such file or directory") in ./sound/firewire/bebob/bebob_command.c
EXN: Sys_error("./sound/firewire/fireworks/packets-buffer.h: No such file or directory") in ./sound/firewire/fireworks/fireworks_transaction.c
--- ./fs/dlm/requestqueue.c
--- ./drivers/w1/w1_netlink.c
--- ./drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
--- ./drivers/platform/surface/surface_acpi_notify.c
--- ./net/nfc/hci/hcp.c
--- ./drivers/s390/net/qeth_l2_main.c

real    8m50.993s
...

With the latest from git, it additionally misses "fs/dlm/requestqueue.c",
(but it takes half the time):

$ time ~/.local/bin/spatch $ARGS --recursive-includes --jobs 36 --chunksize 1 --dir . | grep ^---
6594 files match
--- ./drivers/w1/w1_netlink.c
--- ./drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
--- ./drivers/platform/surface/surface_acpi_notify.c
--- ./net/nfc/hci/hcp.c
--- ./drivers/s390/net/qeth_l2_main.c

real    4m39.263s
...

If I run it _not_ in parallel it is obviously MUCH slower, but ends up
finding _no_ hits at all!

$ $ time spatch $ARGS --recursive-includes --dir . | tee /tmp/slow.log | grep ^---
6594 files match
EXN: Sys_error("./sound/firewire/bebob/lib.h: No such file or directory") in ./sound/firewire/bebob/bebob_command.c
EXN: Sys_error("./sound/firewire/fireworks/packets-buffer.h: No such file or directory") in ./sound/firewire/fireworks/fireworks_command.c

real    66m49.620s
...

Do you have any idea what is going on?

-- 
Kees Cook

^ permalink raw reply	[flat|nested] 7+ messages in thread