Re: [PATCH] Teach "git add" and friends to be paranoid

[Dmitry Potapov <dpotapov@gmail.com>]

On Sun, Feb 21, 2010 at 11:32:10AM -0800, Junio C Hamano wrote:
> 
> 32*1024 sounds like a better cut-off to me.  After that, doubling the size
> does not get comparable gain, and numbers get unstable (notice the glitch
> around 256kB).

The reduction of speed-up after 32Kb is most likely due to L1 cache
size, which is 32Kb data per core on Core 2, and L2 cache is shared
among cores and is considerably slow. I have run my test a few more
times, and here are results:

   1 - 39.25%
   2 - 30.00%
   4 - 17.79%
   8 - 11.76%
  16 - 7.58%
  32 - 5.38%
  64 - 3.89%
 128 - 2.87%
 256 - 2.31%
 512 - 2.92%
1024 - 1.14%

and here is one more re-run starting with 32Kb:

  32 - 5.38%
  64 - 3.89%
 128 - 2.29%
 256 - 2.91%
 512 - 2.92%
1024 - 1.14%

If you look at speed-up numbers, you can think that the numbers are
unstable, but in fact, the best time in 5 runs does not differ more
than 0.01s between those trials. But because difference for >=128Kb
is 0.05s or less, the accuracy of the above numbers is less than 25%.
But overall the outcome is clear -- read() is always a winner.

It would be interesting to see what difference Nehalem, which has a
smaller but much faster L2 cache than Core 2. It may perform better
at larger sizes up to 256Kb.

Anyway, based on above data, I believe that the proper cut-off should be
at least 64Kb, because additional 32Kb (from 32Kb to 64Kb) is about of
2.5% of total memory that git consumes anyway, and it gives you speed-up
around 3.5%...



Dmitry