git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jeff King <peff@peff.net>
To: Junio C Hamano <gitster@pobox.com>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 09/12] receive-pack: use oidset to de-duplicate .have lines
Date: Wed, 25 Jan 2017 14:54:25 -0500	[thread overview]
Message-ID: <20170125195425.q4fpvc4ten5mfjgl@sigill.intra.peff.net> (raw)
In-Reply-To: <xmqqefzraqbu.fsf@gitster.mtv.corp.google.com>

On Wed, Jan 25, 2017 at 11:32:05AM -0800, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > If you have an alternate object store with a very large
> > number of refs, the peak memory usage of the sha1_array can
> > grow high, even if most of them are duplicates that end up
> > not being printed at all.
> > ...
> > Signed-off-by: Jeff King <peff@peff.net>
> > ---
> >  builtin/receive-pack.c | 26 ++++++++++++--------------
> >  1 file changed, 12 insertions(+), 14 deletions(-)
> 
> Nice.  
> 
> Incidentally, this also shows that the refnames in alternate ref
> namespace will not matter.  We are to only show just one of many
> anyway (and that name is masked and replaced with ".have").  Perhaps
> we want to do 04/12 without refname?

I kept the refnames there as a "plausible" thing that a callback might
want. But I have ulterior motives. :)

I have another series which uses alternate refs as part of
check_connected(). Since that function calls rev-list, it needs to add
something like "rev-list --alternate-refs". Even check_connected() does
not care about the refnames, it seems like that rev-list option should
have some useful name for each ref. It can make things like "--source"
more sensible, instead of just reporting everything as a blank ".have".

I could be persuaded otherwise, though.  I don't think we pay a huge for
the refnames here. The generating for-each-ref has them either way, and
the receiving side only keeps one in memory at a time. So we are really
only paying to send them across the pipe and parse them, which doesn't
seem extravagant. I didn't actually measure it, though.

-Peff

  reply	other threads:[~2017-01-25 19:54 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-01-24  0:37 [PATCH 0/12] reducing resource usage of for_each_alternate_ref Jeff King
2017-01-24  0:38 ` [PATCH 01/12] for_each_alternate_ref: handle failure from real_pathdup() Jeff King
2017-01-25 18:26   ` Junio C Hamano
2017-01-24  0:39 ` [PATCH 02/12] for_each_alternate_ref: stop trimming trailing slashes Jeff King
2017-01-24  0:40 ` [PATCH 03/12] for_each_alternate_ref: use strbuf for path allocation Jeff King
2017-01-25 18:29   ` Junio C Hamano
2017-01-25 18:40     ` Jeff King
2017-01-24  0:40 ` [PATCH 04/12] for_each_alternate_ref: pass name/oid instead of ref struct Jeff King
2017-01-24  0:44 ` [PATCH 05/12] for_each_alternate_ref: replace transport code with for-each-ref Jeff King
2017-01-25 19:00   ` Junio C Hamano
2017-01-24  0:45 ` [PATCH 06/12] clone: disable save_commit_buffer Jeff King
2017-01-25 19:11   ` Junio C Hamano
2017-01-25 19:27     ` Jeff King
2017-01-25 19:35       ` Jeff King
2017-01-25 21:07         ` Jeff King
2017-01-24  0:45 ` [PATCH 07/12] fetch-pack: cache results of for_each_alternate_ref Jeff King
2017-01-25 19:21   ` Junio C Hamano
2017-01-25 19:47     ` Jeff King
2017-01-24  0:46 ` [PATCH 08/12] add oidset API Jeff King
2017-01-24 20:26   ` Ramsay Jones
2017-01-24 20:35     ` Jeff King
2017-01-24  0:47 ` [PATCH 09/12] receive-pack: use oidset to de-duplicate .have lines Jeff King
2017-01-25 19:32   ` Junio C Hamano
2017-01-25 19:54     ` Jeff King [this message]
2017-01-24  0:47 ` [PATCH 10/12] receive-pack: fix misleading namespace/.have comment Jeff King
2017-01-24  0:48 ` [PATCH 11/12] receive-pack: treat namespace .have lines like alternates Jeff King
2017-01-25 19:51   ` Junio C Hamano
2017-01-25 19:58     ` Jeff King
2017-01-27 17:45     ` Lukas Fleischer
2017-01-27 17:58       ` Jeff King
2017-01-27 20:42         ` Junio C Hamano
2017-01-24  0:48 ` [PATCH 12/12] receive-pack: avoid duplicates between our refs and alternates Jeff King
2017-01-25 20:02   ` Junio C Hamano
2017-01-25 20:05     ` Jeff King
2017-01-24  1:33 ` [PATCH 0/12] reducing resource usage of for_each_alternate_ref Brandon Williams
2017-01-24  2:12   ` Jeff King
2017-02-08 20:52 ` [PATCH v2 0/11] " Jeff King
2017-02-08 20:52   ` [PATCH v2 01/11] for_each_alternate_ref: handle failure from real_pathdup() Jeff King
2017-02-08 20:52   ` [PATCH v2 02/11] for_each_alternate_ref: stop trimming trailing slashes Jeff King
2017-02-08 20:52   ` [PATCH v2 03/11] for_each_alternate_ref: use strbuf for path allocation Jeff King
2017-02-08 20:52   ` [PATCH v2 04/11] for_each_alternate_ref: pass name/oid instead of ref struct Jeff King
2017-02-08 20:53   ` [PATCH v2 05/11] for_each_alternate_ref: replace transport code with for-each-ref Jeff King
2017-02-08 20:53   ` [PATCH v2 06/11] fetch-pack: cache results of for_each_alternate_ref Jeff King
2017-02-08 20:53   ` [PATCH v2 07/11] add oidset API Jeff King
2017-02-08 20:53   ` [PATCH v2 08/11] receive-pack: use oidset to de-duplicate .have lines Jeff King
2017-02-08 20:53   ` [PATCH v2 09/11] receive-pack: fix misleading namespace/.have comment Jeff King
2017-02-08 20:53   ` [PATCH v2 10/11] receive-pack: treat namespace .have lines like alternates Jeff King
2017-02-08 20:53   ` [PATCH v2 11/11] receive-pack: avoid duplicates between our refs and alternates Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170125195425.q4fpvc4ten5mfjgl@sigill.intra.peff.net \
    --to=peff@peff.net \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).