git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Ramsay Jones <ramsay@ramsayjones.plus.com>
To: Jeff King <peff@peff.net>, git@vger.kernel.org
Subject: Re: [PATCH 08/12] add oidset API
Date: Tue, 24 Jan 2017 20:26:40 +0000	[thread overview]
Message-ID: <944ea1f8-8f9c-cc17-02a5-a73cb6565b45@ramsayjones.plus.com> (raw)
In-Reply-To: <20170124004647.3o26ionfq3td2irf@sigill.intra.peff.net>



On 24/01/17 00:46, Jeff King wrote:
> This is similar to many of our uses of sha1-array, but it
> overcomes one limitation of a sha1-array: when you are
> de-duplicating a large input with relatively few unique
> entries, sha1-array uses 20 bytes per non-unique entry.
> Whereas this set will use memory linear in the number of
> unique entries (albeit a few more than 20 bytes due to
> hashmap overhead).
> 
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> This may be overkill. You can get roughly the same thing by making
> actual object structs via lookup_unknown_object(). But see the next
> patch for some comments on that.
> 
>  Makefile |  1 +
>  oidset.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
>  oidset.h | 45 +++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 95 insertions(+)
>  create mode 100644 oidset.c
>  create mode 100644 oidset.h
> 
> diff --git a/Makefile b/Makefile
> index 27afd0f37..e41efc2d8 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -774,6 +774,7 @@ LIB_OBJS += notes-cache.o
>  LIB_OBJS += notes-merge.o
>  LIB_OBJS += notes-utils.o
>  LIB_OBJS += object.o
> +LIB_OBJS += oidset.o
>  LIB_OBJS += pack-bitmap.o
>  LIB_OBJS += pack-bitmap-write.o
>  LIB_OBJS += pack-check.o
> diff --git a/oidset.c b/oidset.c
> new file mode 100644
> index 000000000..6094cff8c
> --- /dev/null
> +++ b/oidset.c
> @@ -0,0 +1,49 @@
> +#include "cache.h"
> +#include "oidset.h"
> +
> +struct oidset_entry {
> +	struct hashmap_entry hash;
> +	struct object_id oid;
> +};
> +
> +int oidset_hashcmp(const void *va, const void *vb,

static int oidset_hashcmp( ...

ATB,
Ramsay Jones


  reply	other threads:[~2017-01-24 20:26 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-01-24  0:37 [PATCH 0/12] reducing resource usage of for_each_alternate_ref Jeff King
2017-01-24  0:38 ` [PATCH 01/12] for_each_alternate_ref: handle failure from real_pathdup() Jeff King
2017-01-25 18:26   ` Junio C Hamano
2017-01-24  0:39 ` [PATCH 02/12] for_each_alternate_ref: stop trimming trailing slashes Jeff King
2017-01-24  0:40 ` [PATCH 03/12] for_each_alternate_ref: use strbuf for path allocation Jeff King
2017-01-25 18:29   ` Junio C Hamano
2017-01-25 18:40     ` Jeff King
2017-01-24  0:40 ` [PATCH 04/12] for_each_alternate_ref: pass name/oid instead of ref struct Jeff King
2017-01-24  0:44 ` [PATCH 05/12] for_each_alternate_ref: replace transport code with for-each-ref Jeff King
2017-01-25 19:00   ` Junio C Hamano
2017-01-24  0:45 ` [PATCH 06/12] clone: disable save_commit_buffer Jeff King
2017-01-25 19:11   ` Junio C Hamano
2017-01-25 19:27     ` Jeff King
2017-01-25 19:35       ` Jeff King
2017-01-25 21:07         ` Jeff King
2017-01-24  0:45 ` [PATCH 07/12] fetch-pack: cache results of for_each_alternate_ref Jeff King
2017-01-25 19:21   ` Junio C Hamano
2017-01-25 19:47     ` Jeff King
2017-01-24  0:46 ` [PATCH 08/12] add oidset API Jeff King
2017-01-24 20:26   ` Ramsay Jones [this message]
2017-01-24 20:35     ` Jeff King
2017-01-24  0:47 ` [PATCH 09/12] receive-pack: use oidset to de-duplicate .have lines Jeff King
2017-01-25 19:32   ` Junio C Hamano
2017-01-25 19:54     ` Jeff King
2017-01-24  0:47 ` [PATCH 10/12] receive-pack: fix misleading namespace/.have comment Jeff King
2017-01-24  0:48 ` [PATCH 11/12] receive-pack: treat namespace .have lines like alternates Jeff King
2017-01-25 19:51   ` Junio C Hamano
2017-01-25 19:58     ` Jeff King
2017-01-27 17:45     ` Lukas Fleischer
2017-01-27 17:58       ` Jeff King
2017-01-27 20:42         ` Junio C Hamano
2017-01-24  0:48 ` [PATCH 12/12] receive-pack: avoid duplicates between our refs and alternates Jeff King
2017-01-25 20:02   ` Junio C Hamano
2017-01-25 20:05     ` Jeff King
2017-01-24  1:33 ` [PATCH 0/12] reducing resource usage of for_each_alternate_ref Brandon Williams
2017-01-24  2:12   ` Jeff King
2017-02-08 20:52 ` [PATCH v2 0/11] " Jeff King
2017-02-08 20:52   ` [PATCH v2 01/11] for_each_alternate_ref: handle failure from real_pathdup() Jeff King
2017-02-08 20:52   ` [PATCH v2 02/11] for_each_alternate_ref: stop trimming trailing slashes Jeff King
2017-02-08 20:52   ` [PATCH v2 03/11] for_each_alternate_ref: use strbuf for path allocation Jeff King
2017-02-08 20:52   ` [PATCH v2 04/11] for_each_alternate_ref: pass name/oid instead of ref struct Jeff King
2017-02-08 20:53   ` [PATCH v2 05/11] for_each_alternate_ref: replace transport code with for-each-ref Jeff King
2017-02-08 20:53   ` [PATCH v2 06/11] fetch-pack: cache results of for_each_alternate_ref Jeff King
2017-02-08 20:53   ` [PATCH v2 07/11] add oidset API Jeff King
2017-02-08 20:53   ` [PATCH v2 08/11] receive-pack: use oidset to de-duplicate .have lines Jeff King
2017-02-08 20:53   ` [PATCH v2 09/11] receive-pack: fix misleading namespace/.have comment Jeff King
2017-02-08 20:53   ` [PATCH v2 10/11] receive-pack: treat namespace .have lines like alternates Jeff King
2017-02-08 20:53   ` [PATCH v2 11/11] receive-pack: avoid duplicates between our refs and alternates Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=944ea1f8-8f9c-cc17-02a5-a73cb6565b45@ramsayjones.plus.com \
    --to=ramsay@ramsayjones.plus.com \
    --cc=git@vger.kernel.org \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).