From: "Justin Tobler via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: Patrick Steinhardt <ps@pks.im>,
Karthik Nayak <karthik.188@gmail.com>,
Han-Wen Nienhuys <hanwenn@gmail.com>,
Justin Tobler <jltobler@gmail.com>
Subject: [PATCH v5 0/3] reftable/stack: use geometric table compaction
Date: Thu, 04 Apr 2024 18:29:26 +0000 [thread overview]
Message-ID: <pull.1683.v5.git.1712255369.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.1683.v4.git.1712103636.gitgitgadget@gmail.com>
Hello again,
This is the fifth version my patch series that refactors the reftable
compaction strategy to instead follow a geometric sequence. Changes compared
to v4:
* To fix some failing tests and conflicts, this patch series now depends on
the ps/pack-refs-auto series which is currently in next.
* Lifted the GIT_TEST_REFTABLE_AUTOCOMPACTION env out of the reftable
library and into the reftable backend code.
Thanks for taking a look!
-Justin
Justin Tobler (3):
reftable/stack: allow disabling of auto-compaction
reftable/stack: add env to disable autocompaction
reftable/stack: use geometric table compaction
refs/reftable-backend.c | 4 ++
reftable/reftable-writer.h | 3 +
reftable/stack.c | 125 +++++++++++++++++++------------------
reftable/stack.h | 4 --
reftable/stack_test.c | 77 ++++++-----------------
t/t0610-reftable-basics.sh | 71 ++++++++++++++++-----
6 files changed, 146 insertions(+), 138 deletions(-)
base-commit: 4b32163adf4863c6df3bb6b43540fa2ca3494e28
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1683%2Fjltobler%2Fjt%2Freftable-geometric-compaction-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1683/jltobler/jt/reftable-geometric-compaction-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/1683
Range-diff vs v4:
-: ----------- > 1: a7011dbc6aa reftable/stack: allow disabling of auto-compaction
1: 2a0421e5f20 ! 2: 7c4fe0e9ec5 reftable/stack: add env to disable autocompaction
@@ Commit message
Signed-off-by: Justin Tobler <jltobler@gmail.com>
- ## reftable/stack.c ##
-@@ reftable/stack.c: int reftable_addition_commit(struct reftable_addition *add)
- if (err)
- goto done;
-
-- if (!add->stack->disable_auto_compact)
-+ if (!add->stack->disable_auto_compact &&
-+ git_env_bool("GIT_TEST_REFTABLE_AUTOCOMPACTION", 1))
- err = reftable_stack_auto_compact(add->stack);
-
- done:
-
- ## reftable/system.h ##
-@@ reftable/system.h: license that can be found in the LICENSE file or at
- #include "tempfile.h"
- #include "hash-ll.h" /* hash ID, sizes.*/
- #include "dir.h" /* remove_dir_recursively, for tests.*/
+ ## refs/reftable-backend.c ##
+@@
+ #include "../reftable/reftable-merged.h"
+ #include "../setup.h"
+ #include "../strmap.h"
+#include "parse.h"
+ #include "refs-internal.h"
- int hash_size(uint32_t id);
+ /*
+@@ refs/reftable-backend.c: static struct ref_store *reftable_be_init(struct repository *repo,
+ refs->write_options.hash_id = repo->hash_algo->format_id;
+ refs->write_options.default_permissions = calc_shared_perm(0666 & ~mask);
++ if (!git_env_bool("GIT_TEST_REFTABLE_AUTOCOMPACTION", 1))
++ refs->write_options.disable_auto_compact = 1;
++
+ /*
+ * Set up the main reftable stack that is hosted in GIT_COMMON_DIR.
+ * This stack contains both the shared and the main worktree refs.
## t/t0610-reftable-basics.sh ##
@@ t/t0610-reftable-basics.sh: test_expect_success 'ref transaction: writes cause auto-compaction' '
2: e0f4d0dbcc1 ! 3: 8f124acf0f8 reftable/stack: use geometric table compaction
@@ reftable/stack_test.c: static void test_empty_add(void)
+
static void test_reftable_stack_auto_compaction(void)
{
- struct reftable_write_options cfg = { 0 };
+ struct reftable_write_options cfg = {
@@ reftable/stack_test.c: static void test_reftable_stack_compaction_concurrent_clean(void)
int stack_test_main(int argc, const char *argv[])
{
@@ t/t0610-reftable-basics.sh: test_expect_success 'ref transaction: writes are syn
EOF
'
+@@ t/t0610-reftable-basics.sh: test_expect_success 'ref transaction: fails gracefully when auto compaction fail
+ done ||
+ exit 1
+ done &&
+- test_line_count = 13 .git/reftable/tables.list
++ test_line_count = 10 .git/reftable/tables.list
+ )
+ '
+
@@ t/t0610-reftable-basics.sh: test_expect_success 'pack-refs: compacts tables' '
test_commit -C repo A &&
@@ t/t0610-reftable-basics.sh: test_expect_success 'pack-refs: compacts tables' '
git -C repo pack-refs &&
ls -1 repo/.git/reftable >table-files &&
+@@ t/t0610-reftable-basics.sh: test_expect_success "$command: auto compaction" '
+ # The tables should have been auto-compacted, and thus auto
+ # compaction should not have to do anything.
+ ls -1 .git/reftable >tables-expect &&
+- test_line_count = 4 tables-expect &&
++ test_line_count = 3 tables-expect &&
+ git $command --auto &&
+ ls -1 .git/reftable >tables-actual &&
+ test_cmp tables-expect tables-actual &&
+@@ t/t0610-reftable-basics.sh: test_expect_success "$command: auto compaction" '
+ git branch B &&
+ git branch C &&
+ rm .git/reftable/*.lock &&
+- test_line_count = 5 .git/reftable/tables.list &&
++ test_line_count = 4 .git/reftable/tables.list &&
+
+ git $command --auto &&
+ test_line_count = 1 .git/reftable/tables.list
@@ t/t0610-reftable-basics.sh: do
umask $umask &&
git init --shared=true repo &&
--
gitgitgadget
next prev parent reply other threads:[~2024-04-04 18:29 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-05 20:03 [PATCH] reftable/stack: use geometric table compaction Justin Tobler via GitGitGadget
2024-03-06 12:30 ` Patrick Steinhardt
2024-03-06 12:37 ` Patrick Steinhardt
2024-03-21 22:48 ` Justin Tobler
2024-03-21 22:40 ` [PATCH v2 0/3] " Justin Tobler via GitGitGadget
2024-03-21 22:40 ` [PATCH v2 1/3] reftable/stack: add env to disable autocompaction Justin Tobler via GitGitGadget
2024-03-22 1:25 ` Patrick Steinhardt
2024-03-21 22:40 ` [PATCH v2 2/3] reftable/stack: use geometric table compaction Justin Tobler via GitGitGadget
2024-03-22 1:25 ` Patrick Steinhardt
2024-03-27 13:24 ` Karthik Nayak
2024-03-21 22:40 ` [PATCH v2 3/3] reftable/segment: make segment end inclusive Justin Tobler via GitGitGadget
2024-03-22 1:25 ` [PATCH v2 0/3] reftable/stack: use geometric table compaction Patrick Steinhardt
2024-04-03 10:13 ` Han-Wen Nienhuys
2024-04-03 10:18 ` Patrick Steinhardt
2024-04-03 15:14 ` Justin Tobler
2024-04-03 16:40 ` Junio C Hamano
2024-03-29 4:16 ` [PATCH v3 " Justin Tobler via GitGitGadget
2024-03-29 4:16 ` [PATCH v3 1/3] reftable/stack: add env to disable autocompaction Justin Tobler via GitGitGadget
2024-03-29 18:25 ` Junio C Hamano
2024-03-29 21:56 ` Junio C Hamano
2024-04-02 7:23 ` Patrick Steinhardt
2024-04-02 17:23 ` Junio C Hamano
2024-03-29 4:16 ` [PATCH v3 2/3] reftable/stack: use geometric table compaction Justin Tobler via GitGitGadget
2024-04-02 7:23 ` Patrick Steinhardt
2024-03-29 4:16 ` [PATCH v3 3/3] reftable/stack: make segment end inclusive Justin Tobler via GitGitGadget
2024-03-29 18:36 ` Junio C Hamano
2024-04-02 7:23 ` Patrick Steinhardt
2024-04-03 0:20 ` [PATCH v4 0/2] reftable/stack: use geometric table compaction Justin Tobler via GitGitGadget
2024-04-03 0:20 ` [PATCH v4 1/2] reftable/stack: add env to disable autocompaction Justin Tobler via GitGitGadget
2024-04-03 0:20 ` [PATCH v4 2/2] reftable/stack: use geometric table compaction Justin Tobler via GitGitGadget
2024-04-03 4:47 ` [PATCH v4 0/2] " Patrick Steinhardt
2024-04-03 11:12 ` Karthik Nayak
2024-04-03 16:56 ` Junio C Hamano
2024-04-04 18:29 ` Justin Tobler via GitGitGadget [this message]
2024-04-04 18:29 ` [PATCH v5 1/3] reftable/stack: allow disabling of auto-compaction Justin Tobler via GitGitGadget
2024-04-08 6:12 ` Patrick Steinhardt
2024-04-04 18:29 ` [PATCH v5 2/3] reftable/stack: add env to disable autocompaction Justin Tobler via GitGitGadget
2024-04-08 6:12 ` Patrick Steinhardt
2024-04-08 16:18 ` Junio C Hamano
2024-04-04 18:29 ` [PATCH v5 3/3] reftable/stack: use geometric table compaction Justin Tobler via GitGitGadget
2024-04-08 6:12 ` [PATCH v5 0/3] " Patrick Steinhardt
2024-04-08 16:17 ` Justin Tobler
2024-04-08 16:16 ` [PATCH v6 " Justin Tobler via GitGitGadget
2024-04-08 16:16 ` [PATCH v6 1/3] reftable/stack: expose option to disable auto-compaction Justin Tobler via GitGitGadget
2024-04-08 16:16 ` [PATCH v6 2/3] reftable/stack: add env to disable autocompaction Justin Tobler via GitGitGadget
2024-04-08 16:16 ` [PATCH v6 3/3] reftable/stack: use geometric table compaction Justin Tobler via GitGitGadget
2024-04-08 16:20 ` [PATCH v6 0/3] " Patrick Steinhardt
2024-04-08 19:12 ` Junio C Hamano
2024-04-03 19:12 ` [PATCH v2 " Junio C Hamano
2024-04-03 19:30 ` Patrick Steinhardt
2024-04-04 5:34 ` Patrick Steinhardt
2024-04-04 18:28 ` Justin Tobler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=pull.1683.v5.git.1712255369.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=git@vger.kernel.org \
--cc=hanwenn@gmail.com \
--cc=jltobler@gmail.com \
--cc=karthik.188@gmail.com \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.