Re: Mercurial 0.3 vs git benchmarks

From: Linus Torvalds <torvalds@osdl.org>
To: Mike Taht <mike.taht@timesys.com>
Cc: Matt Mackall <mpm@selenic.com>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	git@vger.kernel.org
Subject: Re: Mercurial 0.3 vs git benchmarks
Date: Mon, 25 Apr 2005 20:04:54 -0700 (PDT)	[thread overview]
Message-ID: <Pine.LNX.4.58.0504251938210.18901@ppc970.osdl.org> (raw)
In-Reply-To: <426DA7B5.2080204@timesys.com>

On Mon, 25 Apr 2005, Mike Taht wrote:
> 
> One difference is probably - mercurial appears to be using zlib's 
> *default* compression of 6....
> 
> using zlib compression of 9 really impacts git...

I agree that it will hurt for big changes, but since I really do believe 
that most changes are just a couple of files, I don't believe it matters 
for those. 

I forget what the exact numbers were, but I did some timings on plain
"gzip", and it basically said that doing gzip on a medium-sized file was
not that different for -6 and -9. Why? Because most of the overhead was
elsewhere ;)

Oh, well, I just re-created some numbers. This wasn't exactly what I did 
last time I tested it, but it's conceptually the same thing:

	torvalds@ppc970:~> time gzip -9 < v2.6/linux/kernel/sched.c > /dev/null 
	real    0m0.018s
	user    0m0.018s
	sys     0m0.000s

	torvalds@ppc970:~> time gzip -6 < v2.6/linux/kernel/sched.c > /dev/null 
	real    0m0.015s
	user    0m0.013s
	sys     0m0.001s

ie there's a 0.003 second difference, which is certainly noticeable, and
would be hugely noticeable if you did a lot of these. But in my world-view
(which is what git is optimized for), the common case is that you usually
end up compressing maybe five-ten files, so the _compression_ overhead is
not that huge compared to all the other stuff.

But yes, testing git on big changes will test exactly the things that git
isn't optimized for. I think git will normally hold up pretty well (ie it
will still beat anything that isn't designed for speed, and will be
comparable to things that _are_), but it's not what I'm interested in
optimizing for.

That said - these days we can trivially change over to a "zlib -6" 
compression, and nothing should ever notice. So if somebody wants to 
test it, it should be fairly easy to just compare side-by-side: the 
results should be identical.

The easiest test-case is Andrew's 198-patch patch-bomb on linux-kernel a 
few weeks ago: they all apply cleanly to 2.6.12-rc2 (in order), and you 
can use my "dotest" script to automate the test..

			Linus