From: Eric Wong <e@80x24.org>
To: git@vger.kernel.org
Cc: "Jeff King" <peff@peff.net>, "René Scharfe" <l.s.r@web.de>
Subject: [PATCH v2 0/5] optimizations for many alternates
Date: Tue, 29 Jun 2021 20:53:00 +0000 [thread overview]
Message-ID: <20210629205305.7100-1-e@80x24.org> (raw)
In-Reply-To: <20210627024718.25383-1-e@80x24.org>
v2 has better naming for 3/5, fix test for sha256 in 5/5
Thanks to René and Junio for feedback so far.
Eric Wong (5):
speed up alt_odb_usable() with many alternates
avoid strlen via strbuf_addstr in link_alt_odb_entry
make object_directory.loose_objects_subdir_seen a bitmap
oidcpy_with_padding: constify `src' arg
oidtree: a crit-bit tree for odb_loose_cache
Makefile | 3 +
alloc.c | 6 ++
alloc.h | 1 +
cbtree.c | 167 ++++++++++++++++++++++++++++++++++++++++
cbtree.h | 56 ++++++++++++++
hash.h | 2 +-
object-file.c | 69 ++++++++++-------
object-name.c | 28 +++----
object-store.h | 24 +++++-
object.c | 2 +
oidtree.c | 94 ++++++++++++++++++++++
oidtree.h | 29 +++++++
t/helper/test-oidtree.c | 47 +++++++++++
t/helper/test-tool.c | 1 +
t/helper/test-tool.h | 1 +
t/t0069-oidtree.sh | 52 +++++++++++++
16 files changed, 533 insertions(+), 49 deletions(-)
create mode 100644 cbtree.c
create mode 100644 cbtree.h
create mode 100644 oidtree.c
create mode 100644 oidtree.h
create mode 100644 t/helper/test-oidtree.c
create mode 100755 t/t0069-oidtree.sh
Interdiff against v1:
diff --git a/object-file.c b/object-file.c
index d33b84c4a4..6c397fb4f1 100644
--- a/object-file.c
+++ b/object-file.c
@@ -2463,16 +2463,17 @@ struct oidtree *odb_loose_cache(struct object_directory *odb,
{
int subdir_nr = oid->hash[0];
struct strbuf buf = STRBUF_INIT;
- size_t BM_SIZE = sizeof(odb->loose_objects_subdir_seen[0]) * CHAR_BIT;
+ size_t word_bits = bitsizeof(odb->loose_objects_subdir_seen[0]);
+ size_t word_index = subdir_nr / word_bits;
+ size_t mask = 1 << (subdir_nr % word_bits);
uint32_t *bitmap;
- uint32_t bit = 1 << (subdir_nr % BM_SIZE);
if (subdir_nr < 0 ||
- subdir_nr >= ARRAY_SIZE(odb->loose_objects_subdir_seen) * BM_SIZE)
+ subdir_nr >= bitsizeof(odb->loose_objects_subdir_seen))
BUG("subdir_nr out of range");
- bitmap = &odb->loose_objects_subdir_seen[subdir_nr / BM_SIZE];
- if (*bitmap & bit)
+ bitmap = &odb->loose_objects_subdir_seen[word_index];
+ if (*bitmap & mask)
return &odb->loose_objects_cache;
strbuf_addstr(&buf, odb->path);
@@ -2480,7 +2481,7 @@ struct oidtree *odb_loose_cache(struct object_directory *odb,
append_loose_object,
NULL, NULL,
&odb->loose_objects_cache);
- *bitmap |= bit;
+ *bitmap |= mask;
strbuf_release(&buf);
return &odb->loose_objects_cache;
}
diff --git a/t/helper/test-oidtree.c b/t/helper/test-oidtree.c
index 44bb2e7c29..e0da13eea3 100644
--- a/t/helper/test-oidtree.c
+++ b/t/helper/test-oidtree.c
@@ -13,6 +13,7 @@ int cmd__oidtree(int argc, const char **argv)
struct oidtree ot = OIDTREE_INIT;
struct strbuf line = STRBUF_INIT;
int nongit_ok;
+ int algo = GIT_HASH_UNKNOWN;
setup_git_directory_gently(&nongit_ok);
@@ -21,20 +22,21 @@ int cmd__oidtree(int argc, const char **argv)
struct object_id oid;
if (skip_prefix(line.buf, "insert ", &arg)) {
- if (get_oid_hex(arg, &oid))
- die("not a hexadecimal oid: %s", arg);
+ if (get_oid_hex_any(arg, &oid) == GIT_HASH_UNKNOWN)
+ die("insert not a hexadecimal oid: %s", arg);
+ algo = oid.algo;
oidtree_insert(&ot, &oid);
} else if (skip_prefix(line.buf, "contains ", &arg)) {
if (get_oid_hex(arg, &oid))
- die("not a hexadecimal oid: %s", arg);
+ die("contains not a hexadecimal oid: %s", arg);
printf("%d\n", oidtree_contains(&ot, &oid));
} else if (skip_prefix(line.buf, "each ", &arg)) {
- char buf[GIT_SHA1_HEXSZ + 1] = { '0' };
+ char buf[GIT_MAX_HEXSZ + 1] = { '0' };
memset(&oid, 0, sizeof(oid));
memcpy(buf, arg, strlen(arg));
- buf[GIT_SHA1_HEXSZ] = 0;
+ buf[hash_algos[algo].hexsz] = 0;
get_oid_hex_any(buf, &oid);
- oid.algo = GIT_HASH_SHA1;
+ oid.algo = algo;
oidtree_each(&ot, &oid, strlen(arg), print_oid, NULL);
} else if (!strcmp(line.buf, "destroy"))
oidtree_destroy(&ot);
diff --git a/t/t0069-oidtree.sh b/t/t0069-oidtree.sh
index bb4229210c..0594f57c81 100755
--- a/t/t0069-oidtree.sh
+++ b/t/t0069-oidtree.sh
@@ -10,9 +10,9 @@ echoid () {
do
echo "$1"
shift
- done | awk -v prefix="$prefix" '{
+ done | awk -v prefix="$prefix" -v ZERO_OID=$ZERO_OID '{
printf("%s%s", prefix, $0);
- need = 40 - length($0);
+ need = length(ZERO_OID) - length($0);
for (i = 0; i < need; i++)
printf("0");
printf "\n";
next prev parent reply other threads:[~2021-06-29 20:53 UTC|newest]
Thread overview: 99+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-24 0:58 [PATCH] speed up alt_odb_usable() with many alternates Eric Wong
2021-06-27 2:47 ` [PATCH 0/5] optimizations for many odb alternates Eric Wong
2021-06-27 2:47 ` [PATCH 1/5] speed up alt_odb_usable() with many alternates Eric Wong
2021-06-27 2:47 ` [PATCH 2/5] avoid strlen via strbuf_addstr in link_alt_odb_entry Eric Wong
2021-06-27 2:47 ` [PATCH 3/5] make object_directory.loose_objects_subdir_seen a bitmap Eric Wong
2021-06-27 10:23 ` René Scharfe
2021-06-28 23:09 ` Eric Wong
2021-06-27 2:47 ` [PATCH 4/5] oidcpy_with_padding: constify `src' arg Eric Wong
2021-06-27 2:47 ` [PATCH 5/5] oidtree: a crit-bit tree for odb_loose_cache Eric Wong
2021-06-29 14:42 ` Junio C Hamano
2021-06-29 20:17 ` Eric Wong
2021-06-29 20:53 ` Eric Wong [this message]
2021-07-07 23:10 ` [PATCH v3 0/5] optimizations for many alternates Eric Wong
2021-07-07 23:10 ` [PATCH v3 1/5] speed up alt_odb_usable() with " Eric Wong
2021-07-08 0:20 ` Junio C Hamano
2021-07-08 1:14 ` Eric Wong
2021-07-08 4:30 ` Junio C Hamano
2021-07-07 23:10 ` [PATCH v3 2/5] avoid strlen via strbuf_addstr in link_alt_odb_entry Eric Wong
2021-07-08 4:57 ` Junio C Hamano
2021-07-07 23:10 ` [PATCH v3 3/5] make object_directory.loose_objects_subdir_seen a bitmap Eric Wong
2021-07-07 23:10 ` [PATCH v3 4/5] oidcpy_with_padding: constify `src' arg Eric Wong
2021-07-07 23:10 ` [PATCH v3 5/5] oidtree: a crit-bit tree for odb_loose_cache Eric Wong
2021-06-29 20:53 ` [PATCH v2 1/5] speed up alt_odb_usable() with many alternates Eric Wong
2021-07-03 10:05 ` René Scharfe
2021-07-04 9:02 ` René Scharfe
2021-07-06 23:01 ` Eric Wong
2021-06-29 20:53 ` [PATCH v2 2/5] avoid strlen via strbuf_addstr in link_alt_odb_entry Eric Wong
2021-06-29 20:53 ` [PATCH v2 3/5] make object_directory.loose_objects_subdir_seen a bitmap Eric Wong
2021-06-29 20:53 ` [PATCH v2 4/5] oidcpy_with_padding: constify `src' arg Eric Wong
2021-06-29 20:53 ` [PATCH v2 5/5] oidtree: a crit-bit tree for odb_loose_cache Eric Wong
2021-07-04 9:02 ` René Scharfe
2021-07-06 23:21 ` Eric Wong
2021-07-04 9:32 ` Ævar Arnfjörð Bjarmason
2021-07-07 23:12 ` Eric Wong
2021-08-06 15:31 ` Andrzej Hunt
2021-08-06 17:53 ` René Scharfe
2021-08-07 22:49 ` Eric Wong
2021-08-09 1:35 ` Carlo Arenas
2021-08-09 1:38 ` [PATCH/RFC 0/3] pedantic errors in next Carlo Marcelo Arenas Belón
2021-08-09 1:38 ` [PATCH/RFC 1/3] oidtree: avoid nested struct oidtree_node Carlo Marcelo Arenas Belón
2021-08-09 1:38 ` [PATCH/RFC 2/3] object-store: avoid extra ';' from KHASH_INIT Carlo Marcelo Arenas Belón
2021-08-09 15:53 ` Junio C Hamano
2021-08-09 1:38 ` [PATCH/RFC 3/3] ci: run a pedantic build as part of the GitHub workflow Carlo Marcelo Arenas Belón
2021-08-09 10:50 ` Bagas Sanjaya
2021-08-09 22:03 ` Carlo Arenas
2021-08-09 14:56 ` Phillip Wood
2021-08-09 22:48 ` Carlo Arenas
2021-08-10 15:24 ` Phillip Wood
2021-08-10 18:25 ` Junio C Hamano
2021-08-30 11:36 ` Ævar Arnfjörð Bjarmason
2021-08-31 20:28 ` Carlo Arenas
2021-08-31 20:51 ` Ævar Arnfjörð Bjarmason
2021-08-31 23:54 ` Carlo Arenas
2021-09-01 1:52 ` Jeff King
2021-09-01 17:55 ` Junio C Hamano
2021-08-30 11:40 ` Ævar Arnfjörð Bjarmason
2021-09-01 9:19 ` [RFC PATCH v2 0/4] developer: support pedantic Carlo Marcelo Arenas Belón
2021-09-01 9:19 ` [RFC PATCH v2 1/4] developer: retire USE_PARENS_AROUND_GETTEXT_N support Carlo Marcelo Arenas Belón
2021-09-01 9:19 ` [RFC PATCH v2 2/4] developer: enable pedantic by default Carlo Marcelo Arenas Belón
2021-09-01 9:19 ` [RFC PATCH v2 3/4] developer: add an alternative script for detecting broken N_() Carlo Marcelo Arenas Belón
2021-09-01 9:19 ` [RFC PATCH v2 4/4] developer: move detect-compiler out of the main directory Carlo Marcelo Arenas Belón
2021-09-01 10:10 ` [RFC PATCH v2 0/4] developer: support pedantic Jeff King
2021-09-01 11:25 ` [PATCH] gettext: remove optional non-standard parens in N_() definition Ævar Arnfjörð Bjarmason
2021-09-01 17:31 ` Eric Sunshine
2021-09-02 9:13 ` Jeff King
2021-09-02 19:19 ` Junio C Hamano
2021-09-01 11:27 ` [RFC PATCH v2 0/4] developer: support pedantic Ævar Arnfjörð Bjarmason
2021-09-01 18:03 ` Carlo Arenas
2021-09-03 17:02 ` [PATCH v3 0/3] support pedantic in developer mode Carlo Marcelo Arenas Belón
2021-09-03 17:02 ` [PATCH v3 1/3] gettext: remove optional non-standard parens in N_() definition Carlo Marcelo Arenas Belón
2021-09-10 15:39 ` Ævar Arnfjörð Bjarmason
2021-09-03 17:02 ` [PATCH v3 2/3] win32: allow building with pedantic mode enabled Carlo Marcelo Arenas Belón
2021-09-03 18:47 ` René Scharfe
2021-09-03 20:13 ` Carlo Marcelo Arenas Belón
2021-09-03 20:32 ` Junio C Hamano
2021-09-03 20:38 ` René Scharfe
2021-09-04 9:37 ` René Scharfe
2021-09-04 14:42 ` Carlo Arenas
2021-09-27 23:04 ` Jonathan Tan
2021-09-28 0:30 ` Carlo Arenas
2021-09-28 16:50 ` Jonathan Tan
2021-09-28 17:37 ` Junio C Hamano
2021-09-28 20:16 ` Jonathan Tan
2021-09-29 1:00 ` Carlo Arenas
2021-09-29 15:55 ` Junio C Hamano
2021-09-03 17:02 ` [PATCH v3 3/3] developer: enable pedantic by default Carlo Marcelo Arenas Belón
2021-09-05 7:54 ` [PATCH v3 0/3] support pedantic in developer mode Ævar Arnfjörð Bjarmason
2021-08-09 16:44 ` [PATCH/RFC 0/3] pedantic errors in next Junio C Hamano
2021-08-09 20:10 ` Eric Wong
2021-08-10 6:16 ` Carlo Marcelo Arenas Belón
2021-08-10 19:30 ` René Scharfe
2021-08-10 23:49 ` Carlo Arenas
2021-08-11 0:57 ` Carlo Arenas
2021-08-11 14:57 ` René Scharfe
2021-08-11 17:20 ` Junio C Hamano
2021-08-10 18:59 ` [PATCH v2 5/5] oidtree: a crit-bit tree for odb_loose_cache René Scharfe
2021-08-10 19:40 ` René Scharfe
2021-08-14 20:00 ` [PATCH] oidtree: avoid unaligned access to crit-bit tree René Scharfe
2021-08-16 19:11 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210629205305.7100-1-e@80x24.org \
--to=e@80x24.org \
--cc=git@vger.kernel.org \
--cc=l.s.r@web.de \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).