Re: [PATCH v2 4/8] scripts: add glimpse.sh for indexing the kernel

From: "Luis R. Rodriguez" <mcgrof@kernel.org>
To: Julia Lawall <julia.lawall@lip6.fr>
Cc: "Luis R. Rodriguez" <mcgrof@kernel.org>,
	Gilles.Muller@lip6.fr, nicolas.palix@imag.fr, mmarek@suse.com,
	linux-kernel@vger.kernel.org, akpm@linux-foundation.org,
	gregkh@linuxfoundation.org, markivx@codeaurora.org,
	stephen.boyd@linaro.org, zohar@linux.vnet.ibm.com,
	broonie@kernel.org, ming.lei@canonical.com, tiwai@suse.de,
	johannes@sipsolutions.net, chunkeey@googlemail.com,
	hauke@hauke-m.de, jwboyer@fedoraproject.org,
	dmitry.torokhov@gmail.com, dwmw2@infradead.org, jslaby@suse.com,
	torvalds@linux-foundation.org, cocci@systeme.lip6.fr
Subject: Re: [PATCH v2 4/8] scripts: add glimpse.sh for indexing the kernel
Date: Sat, 18 Jun 2016 03:22:43 +0200	[thread overview]
Message-ID: <20160618012243.GD11948@wotan.suse.de> (raw)
In-Reply-To: <alpine.DEB.2.10.1606171732480.3164@hadrien>

On Fri, Jun 17, 2016 at 05:35:26PM +0200, Julia Lawall wrote:
> On Fri, 17 Jun 2016, Luis R. Rodriguez wrote:
> 
> > On Fri, Jun 17, 2016 at 11:44:26AM +0200, Julia Lawall wrote:
> > > I'm not sure that this is worth it.  It adds a dependency on a tool that
> > > seems not to be well maintained.  In terms of Coccinelle, I'm not sure
> > > that it gives a big benefit.
> > >
> > > Attached is a graph showing the file selection time for Coccinelle for a
> > > selection of fairly complex semantic patches.  Coccigrep is just a
> > > line-by-line regexp search implemented in ocaml, gitgrep uses git grep.
> > > In most cases, glimpse is clearly faster.
> > >
> > > On the other hand, it seems that glimpse often selects more files.
> > > Sometimes a few more, eg 16 vs 14, and sometimes quite a lot more, eg 538
> > > vs 236.  I suspect that this is because glimpse considers _ to be a space,
> > > and thus it can have many false positives.  There are, however, a few
> > > cases where glimpse also selects fewer files.
> > >
> > > The file processing time (ie parsing the file, searching for, matches of
> > > the semantic patch in the file, and performing the transformation) is
> > > normally much higher than the file selection time.
> > >
> > > So it seems that git grep is currently a better option for the kernel.
> >
> > Great, thanks, consider this patch dropped, do we want the heuristics
> > for the cache index in place though or should I drop that as well ?
> 
> I assume you mean this patch:
> 
> [PATCH v2 3/8] coccicheck: add indexing enhancement options
> 
> I think it should be dropped.  It adds complexity and git grep works
> pretty well.

Hmm but coccicheck does not make use of --git-grep even.

> If people want to use something else, they can use SPARGS,
> or a .cocciconfig file, eg:
> 
> [spatch]
> options = --use-glimpse

Neat will these be used last and thus override anything? If so, what about just
adding an upstream .cocciconfig with --use-gitgrep -- only issue then is
what if a user wants to use idutils ? How do we let them override?

  Luis