All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jeff Hostetler <git@jeffhostetler.com>
To: Jeff King <peff@peff.net>,
	Johannes Schindelin <johannes.schindelin@gmx.de>
Cc: git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>,
	Jeff Hostetler <jeffhost@microsoft.com>
Subject: Re: [PATCH 0/5] A series of performance enhancements in the memihash and name-cache area
Date: Wed, 15 Feb 2017 09:27:53 -0500	[thread overview]
Message-ID: <16b1259c-4cdc-8f4d-db47-d724386a3d2b@jeffhostetler.com> (raw)
In-Reply-To: <20170214220332.753i4tgclm62er4f@sigill.intra.peff.net>



On 2/14/2017 5:03 PM, Jeff King wrote:
> On Tue, Feb 14, 2017 at 12:31:46PM +0100, Johannes Schindelin wrote:
>
>> On Windows, calls to memihash() and maintaining the istate.name_hash and
>> istate.dir_hash HashMaps take significant time on very large
>> repositories. This series of changes reduces the overall time taken for
>> various operations by reducing the number calls to memihash(), moving
>> some of them into multi-threaded code, and etc.
>>
>> Note: one commenter in https://github.com/git-for-windows/git/pull/964
>> pointed out that memihash() only handles ASCII correctly. That is true.
>> And fixing this is outside the purview of this patch series.
> Out of curiosity, do you have numbers? Bonus points if the speedup can
> be shown via a t/perf script.
>
> We have a read-cache perf-test already, but I suspect you'd want
> something more like "git status" or "ls-files -o" that calls into
> read_directory().

I have some informal numbers in a spreadsheet.  I was seeing
a 8-9% speed up on a status on my gigantic repo.

I'll try to put together a before/after perf-test to better demonstrate 
this.

Jeff


  reply	other threads:[~2017-02-15 14:27 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-02-14 11:31 [PATCH 0/5] A series of performance enhancements in the memihash and name-cache area Johannes Schindelin
2017-02-14 11:31 ` [PATCH 1/5] name-hash: eliminate duplicate memihash call Johannes Schindelin
2017-02-14 11:32 ` [PATCH 2/5] hashmap: allow memihash computation to be continued Johannes Schindelin
2017-02-18  5:35   ` Junio C Hamano
2017-02-20 12:43     ` Johannes Schindelin
2017-02-20 20:27       ` Junio C Hamano
2017-02-14 11:32 ` [PATCH 3/5] name-hash: precompute hash values during preload-index Johannes Schindelin
2017-02-18  5:47   ` Junio C Hamano
2017-02-19  0:19     ` Jeff Hostetler
2017-02-19 21:45       ` Junio C Hamano
2017-02-14 11:32 ` [PATCH 4/5] name-hash: specify initial size for istate.dir_hash table Johannes Schindelin
2017-02-14 11:32 ` [PATCH 5/5] name-hash: remember previous dir_entry during lazy_init_name_hash Johannes Schindelin
2017-02-14 22:03 ` [PATCH 0/5] A series of performance enhancements in the memihash and name-cache area Jeff King
2017-02-15 14:27   ` Jeff Hostetler [this message]
2017-02-15 16:44     ` Jeff King
2017-02-18  5:56       ` Junio C Hamano
2017-02-19  0:02         ` Jeff Hostetler
2017-02-18  5:58     ` Junio C Hamano
2017-02-18  6:29       ` Jeff King
2017-02-18 20:48         ` Junio C Hamano
2017-02-18 23:52           ` Jeff Hostetler
2017-02-19 21:50             ` Junio C Hamano
2017-03-02 21:11     ` Junio C Hamano
2017-03-02 21:18       ` Jeff Hostetler
2017-03-02 21:40         ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=16b1259c-4cdc-8f4d-db47-d724386a3d2b@jeffhostetler.com \
    --to=git@jeffhostetler.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jeffhost@microsoft.com \
    --cc=johannes.schindelin@gmx.de \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.