Re: UNLEAK(), leak checking in the default tests etc.

[Andrzej Hunt <andrzej@ahunt.org>]

On 10/06/2021 18:36, Jeff King wrote:
> On Thu, Jun 10, 2021 at 05:32:41PM +0200, Andrzej Hunt wrote:
> 
>>> I do think it would be worth splitting out ASan from leak-checking. The
>>> whole suite should run clean with regular ASan already, and we'd want to
>>> find regressions there even in the tests that aren't leak-clean. I do
>>> periodic ASan runs already; the main argument against doing it for every
>>> CI run is just that's a lot more CPU. But maybe not enough to be
>>> prohibitive? It's probably still way cheaper than running the test suite
>>> on Windows.
>>
>> I've been running tests with ASAN in the Github Actions environment, and a
>> single run takes just over 30 minutes [1] - which I believe is similar to
>> the normal test jobs (they do run the test suite twice in that time I
>> think).
>>
>> I've been doing the same with UBSAN, and that's even faster at 15-20 minutes
>> [2]. However I get the impression that ASAN issues are both more common (at
>> least on seen), and more impactful - so I would argue that ASAN should be
>> prioritised if there's spare capacity. (I have no idea if ASAN+UBSAN can be
>> combined, but I suspect that doing so would make the tests slower?)
> 
> I routinely do SANITIZE=address,undefined since they are both useful
> (and we do not trigger either in the current test suite). I never
> measured the time of their combined use versus just one, but surely it's
> faster the two-at-once approach is faster than running the test suite
> twice.

I'm seeing 33 minutes for SANITIZE=address,undefined - which is no 
slower than SANITIZE=address by itself (disclaimer: it's only one 
measurement):
https://github.com/ahunt/git/runs/2795642716?check_suite_focus=true
(The job's name is wrong but if you look in the logs you can confirm 
that it's using address+undefined.)

The usual linux and mac test-jobs are actually from 24 to 30 minutes 
(the numbers seem a bit variable) - with the exception of one faster 10 
minute job:
https://github.com/git/git/actions/runs/925771097
vs
https://github.com/git/git/actions/runs/927729395

So to summarise: adding an ASAN+UBSAN job would make things a bit 
slower, but not a huge amount slower.