[TOPIC 12/17] Test harness improvements

["James Ramsay" <james@jramsay.com.au>]

1. Jonathan N: Test harness is an important part of the development 
process, shapes what kinds of tests people write. What can we improve?

2. Peff: I love our test harness

3. Brian: It’s amazing for integration tests. For our C code it’s a 
lot harder to do unit tests. We sometimes have a portability issue, 
about POSIX shell vs others.

4. ZJ: I like how it also acts as a piece of documentation.

5. Jonathan N: If we had more unit tests: if I am working on refs, I 
might like to run all tests related to that. And now we have this lack 
of dependency graph between this

6. Peff: I’m super nervous about that. Tools like code coverage could 
do this. But I’ve seen cases where all new tests are green, and tests 
in the area I expected they succeed. But at some far corner it seems to 
fail. So you’re optimizing for speed, might be losing in correctness. 
I’m biased because I can run all tests on my computer in 1 minute. But 
for Windows this doesn’t seem to work that.

7. Peff: We can spend time on speeding up things, making it better 
parallelized for example. I’ll send some patches out on this.

8. Jonathan N: Really nice contribution to Git by David Barr, whose 
background was as a Java developer and thus the code was written in a 
Java way with clear API boundaries and unit tests.

9. Brian: Yes if your function is doing too much, it should be split up 
making it possible to test the separate pieces and then have a function 
that calls those and tests the end result.