All of lore.kernel.org
 help / color / mirror / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: Derrick Stolee <stolee@gmail.com>
Subject: [PATCH 11/15] builtin/pack-refs: introduce new "--auto" flag
Date: Mon, 18 Mar 2024 11:53:13 +0100	[thread overview]
Message-ID: <8727f08bab8459ca608915a6320c817711742a87.1710706118.git.ps@pks.im> (raw)
In-Reply-To: <cover.1710706118.git.ps@pks.im>

[-- Attachment #1: Type: text/plain, Size: 5609 bytes --]

Calling git-pack-refs(1) will unconditionally cause it to pack all
requested refs regardless of the current state of the ref database. For
example:

  - With the "files" backend we will end up rewriting the complete
    "packed-refs" file even if only a single ref would require
    compaction.

  - With the "reftable" backend we will end up always compacting all
    tables into a single table.

This behaviour can be completely unnecessary depending on the backend
and is thus wasteful.

With the introduction of the `PACK_REFS_AUTO` flag in the preceding
commit we can improve this and let the backends decide for themselves
whether to pack refs in the first place. Expose this functionality via a
new "--auto" flag in git-pack-refs(1), which mirrors the same flag in
both git-gc(1) and git-maintenance(1).

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Documentation/git-pack-refs.txt | 15 ++++++++++++++-
 builtin/pack-refs.c             |  3 ++-
 t/t0601-reffiles-pack-refs.sh   |  7 +++++++
 t/t0610-reftable-basics.sh      | 34 +++++++++++++++++++++++++++++++++
 4 files changed, 57 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-pack-refs.txt b/Documentation/git-pack-refs.txt
index 284956acb3..2dcabaf74c 100644
--- a/Documentation/git-pack-refs.txt
+++ b/Documentation/git-pack-refs.txt
@@ -8,7 +8,7 @@ git-pack-refs - Pack heads and tags for efficient repository access
 SYNOPSIS
 --------
 [verse]
-'git pack-refs' [--all] [--no-prune] [--include <pattern>] [--exclude <pattern>]
+'git pack-refs' [--all] [--no-prune] [--auto] [--include <pattern>] [--exclude <pattern>]
 
 DESCRIPTION
 -----------
@@ -60,6 +60,19 @@ with many branches of historical interests.
 The command usually removes loose refs under `$GIT_DIR/refs`
 hierarchy after packing them.  This option tells it not to.
 
+--auto::
+
+Pack refs as needed depending on the current state of the ref database. The
+behavior depends on the ref format used by the repository and may change in the
+future.
++
+	- "files": No special handling for `--auto` has been implemented.
++
+	- "reftable": Tables are compacted such that they form a geometric
+	  sequence. For two tables N and N+1, where N+1 is newer, this
+	  maintains the property that N is at least twice as big as N+1. Only
+	  tables that violate this property are compacted.
+
 --include <pattern>::
 
 Pack refs based on a `glob(7)` pattern. Repetitions of this option
diff --git a/builtin/pack-refs.c b/builtin/pack-refs.c
index ea2baeec76..db40825666 100644
--- a/builtin/pack-refs.c
+++ b/builtin/pack-refs.c
@@ -7,7 +7,7 @@
 #include "revision.h"
 
 static char const * const pack_refs_usage[] = {
-	N_("git pack-refs [--all] [--no-prune] [--include <pattern>] [--exclude <pattern>]"),
+	N_("git pack-refs [--all] [--no-prune] [--auto] [--include <pattern>] [--exclude <pattern>]"),
 	NULL
 };
 
@@ -28,6 +28,7 @@ int cmd_pack_refs(int argc, const char **argv, const char *prefix)
 	struct option opts[] = {
 		OPT_BOOL(0, "all",   &pack_all, N_("pack everything")),
 		OPT_BIT(0, "prune", &pack_refs_opts.flags, N_("prune loose refs (default)"), PACK_REFS_PRUNE),
+		OPT_BIT(0, "auto", &pack_refs_opts.flags, N_("auto-pack refs as needed"), PACK_REFS_AUTO),
 		OPT_STRING_LIST(0, "include", pack_refs_opts.includes, N_("pattern"),
 			N_("references to include")),
 		OPT_STRING_LIST(0, "exclude", &option_excluded_refs, N_("pattern"),
diff --git a/t/t0601-reffiles-pack-refs.sh b/t/t0601-reffiles-pack-refs.sh
index b1cf587347..219a495451 100755
--- a/t/t0601-reffiles-pack-refs.sh
+++ b/t/t0601-reffiles-pack-refs.sh
@@ -164,6 +164,13 @@ test_expect_success 'test --exclude takes precedence over --include' '
 	git pack-refs --include "refs/heads/pack*" --exclude "refs/heads/pack*" &&
 	test -f .git/refs/heads/dont_pack5'
 
+test_expect_success '--auto packs and prunes refs as usual' '
+	git branch auto &&
+	test_path_is_file .git/refs/heads/auto &&
+	git pack-refs --auto --all &&
+	test_path_is_missing .git/refs/heads/auto
+'
+
 test_expect_success 'see if up-to-date packed refs are preserved' '
 	git branch q &&
 	git pack-refs --all --prune &&
diff --git a/t/t0610-reftable-basics.sh b/t/t0610-reftable-basics.sh
index a53d1dc493..6de7529575 100755
--- a/t/t0610-reftable-basics.sh
+++ b/t/t0610-reftable-basics.sh
@@ -387,6 +387,40 @@ test_expect_success 'pack-refs: compaction raises locking errors' '
 	test_cmp expect err
 '
 
+test_expect_success 'pack-refs: auto compaction' '
+	test_when_finished "rm -rf repo" &&
+	git init repo &&
+	(
+		cd repo &&
+
+		test_commit A &&
+
+		# The tables should have been auto-compacted, and thus auto
+		# compaction should not have to do anything.
+		ls -1 .git/reftable >tables-expect &&
+		test_line_count = 4 tables-expect &&
+		git pack-refs --auto &&
+		ls -1 .git/reftable >tables-actual &&
+		test_cmp tables-expect tables-actual &&
+
+		# Lock all tables write some refs. Auto-compaction will be
+		# unable to compact tables and thus fails gracefully, leaving
+		# the stack in a sub-optimal state.
+		ls .git/reftable/*.ref |
+		while read table
+		do
+			touch "$table.lock" || exit 1
+		done &&
+		git branch B &&
+		git branch C &&
+		rm .git/reftable/*.lock &&
+		test_line_count = 5 .git/reftable/tables.list &&
+
+		git pack-refs --auto &&
+		test_line_count = 1 .git/reftable/tables.list
+	)
+'
+
 test_expect_success 'pack-refs: prunes stale tables' '
 	test_when_finished "rm -rf repo" &&
 	git init repo &&
-- 
2.44.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

  parent reply	other threads:[~2024-03-18 10:53 UTC|newest]

Thread overview: 49+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-03-18 10:52 [PATCH 00/15] refs: introduce `--auto` to pack refs as needed Patrick Steinhardt
2024-03-18 10:52 ` [PATCH 01/15] reftable/stack: fix error handling in `reftable_stack_init_addition()` Patrick Steinhardt
2024-03-20 21:50   ` Karthik Nayak
2024-03-25  9:10     ` Patrick Steinhardt
2024-03-18 10:52 ` [PATCH 02/15] reftable/error: discern locked/outdated errors Patrick Steinhardt
2024-03-18 10:52 ` [PATCH 03/15] reftable/stack: use error codes when locking fails during compaction Patrick Steinhardt
2024-03-20 22:14   ` Karthik Nayak
2024-03-25  9:10     ` Patrick Steinhardt
2024-03-18 10:52 ` [PATCH 04/15] reftable/stack: gracefully handle failed auto-compaction due to locks Patrick Steinhardt
2024-03-20 22:22   ` Karthik Nayak
2024-03-25  9:10     ` Patrick Steinhardt
2024-03-18 10:52 ` [PATCH 05/15] refs/reftable: print errors on compaction failure Patrick Steinhardt
2024-03-18 10:52 ` [PATCH 06/15] t/helper: drop pack-refs wrapper Patrick Steinhardt
2024-03-18 10:52 ` [PATCH 07/15] refs: move `struct pack_refs_opts` to where it's used Patrick Steinhardt
2024-03-18 10:52 ` [PATCH 08/15] refs: remove `PACK_REFS_ALL` flag Patrick Steinhardt
2024-03-18 10:53 ` [PATCH 09/15] refs/reftable: expose auto compaction via new flag Patrick Steinhardt
2024-03-18 10:53 ` [PATCH 10/15] builtin/pack-refs: release allocated memory Patrick Steinhardt
2024-03-20 23:23   ` Karthik Nayak
2024-03-25  9:10     ` Patrick Steinhardt
2024-03-18 10:53 ` Patrick Steinhardt [this message]
2024-03-18 10:53 ` [PATCH 12/15] builtin/gc: move `struct maintenance_run_opts` Patrick Steinhardt
2024-03-18 10:53 ` [PATCH 13/15] t6500: extract objects with "17" prefix Patrick Steinhardt
2024-03-18 10:53 ` [PATCH 14/15] builtin/gc: forward git-gc(1)'s `--auto` flag when packing refs Patrick Steinhardt
2024-03-20 23:56   ` Karthik Nayak
2024-03-25  9:10     ` Patrick Steinhardt
2024-03-18 10:53 ` [PATCH 15/15] builtin/gc: pack refs when using `git maintenance run --auto` Patrick Steinhardt
2024-03-20 23:59   ` Karthik Nayak
2024-03-25  9:10     ` Patrick Steinhardt
2024-03-20 19:30 ` [PATCH 00/15] refs: introduce `--auto` to pack refs as needed Karthik Nayak
2024-03-25  9:10   ` Patrick Steinhardt
2024-03-25 10:02 ` [PATCH v2 " Patrick Steinhardt
2024-03-25 10:02   ` [PATCH v2 01/15] reftable/stack: fix error handling in `reftable_stack_init_addition()` Patrick Steinhardt
2024-03-25 10:02   ` [PATCH v2 02/15] reftable/error: discern locked/outdated errors Patrick Steinhardt
2024-03-25 10:02   ` [PATCH v2 03/15] reftable/stack: use error codes when locking fails during compaction Patrick Steinhardt
2024-03-25 10:02   ` [PATCH v2 04/15] reftable/stack: gracefully handle failed auto-compaction due to locks Patrick Steinhardt
2024-03-25 11:22     ` Patrick Steinhardt
2024-03-25 10:02   ` [PATCH v2 05/15] refs/reftable: print errors on compaction failure Patrick Steinhardt
2024-03-25 10:02   ` [PATCH v2 06/15] t/helper: drop pack-refs wrapper Patrick Steinhardt
2024-03-25 10:03   ` [PATCH v2 07/15] refs: move `struct pack_refs_opts` to where it's used Patrick Steinhardt
2024-03-25 10:03   ` [PATCH v2 08/15] refs: remove `PACK_REFS_ALL` flag Patrick Steinhardt
2024-03-25 10:03   ` [PATCH v2 09/15] refs/reftable: expose auto compaction via new flag Patrick Steinhardt
2024-03-25 10:03   ` [PATCH v2 10/15] builtin/pack-refs: release allocated memory Patrick Steinhardt
2024-03-25 10:03   ` [PATCH v2 11/15] builtin/pack-refs: introduce new "--auto" flag Patrick Steinhardt
2024-03-25 10:03   ` [PATCH v2 12/15] builtin/gc: move `struct maintenance_run_opts` Patrick Steinhardt
2024-03-25 10:03   ` [PATCH v2 13/15] t6500: extract objects with "17" prefix Patrick Steinhardt
2024-03-25 10:03   ` [PATCH v2 14/15] builtin/gc: forward git-gc(1)'s `--auto` flag when packing refs Patrick Steinhardt
2024-03-25 10:03   ` [PATCH v2 15/15] builtin/gc: pack refs when using `git maintenance run --auto` Patrick Steinhardt
2024-03-25 11:23   ` [PATCH v2 00/15] refs: introduce `--auto` to pack refs as needed Karthik Nayak
2024-03-25 16:56     ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8727f08bab8459ca608915a6320c817711742a87.1710706118.git.ps@pks.im \
    --to=ps@pks.im \
    --cc=git@vger.kernel.org \
    --cc=stolee@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.