* [PATCH 0/4] rebase: update branches in multi-part topic @ 2022-06-03 13:37 Derrick Stolee via GitGitGadget 2022-06-03 13:37 ` [PATCH 1/4] log-tree: create for_each_decoration() Derrick Stolee via GitGitGadget ` (7 more replies) 0 siblings, 8 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-03 13:37 UTC (permalink / raw) To: git; +Cc: gitster, johannes.schindelin, me, Derrick Stolee This is a feature I've wanted for quite a while. When working on the sparse index topic, I created a long RFC that actually broke into three topics for full review upstream. These topics were sequential, so any feedback on an earlier one required updates to the later ones. I would work on the full feature and use interactive rebase to update the full list of commits. However, I would need to update the branches pointing to those sub-topics. This series adds a new --update-refs option to 'git rebase' (along with a rebase.updateRefs config option) that adds 'git update-ref' commands into the TODO list. This is powered by the commit decoration machinery. As an example, here is my in-progress bundle URI RFC split into subtopics as they appear during the TODO list of a git rebase -i --update-refs: pick 2d966282ff3 docs: document bundle URI standard pick 31396e9171a remote-curl: add 'get' capability pick 54c6ab70f67 bundle-uri: create basic file-copy logic pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// pick 6adaf842684 fetch: add --bundle-uri option pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration exec git update-ref refs/heads/bundle-redo/fetch HEAD 6c5840ed77e1bc41c1fe6fb7c894ceede1b8d730 pick 1e3f6546632 clone: add --bundle-uri option pick 9e4a6fe9b68 clone: --bundle-uri cannot be combined with --depth exec git update-ref refs/heads/bundle-redo/clone HEAD 9e4a6fe9b68a8455b427c9ac8cdbff30c96653b4 pick 5451cb6599c bundle-uri: create bundle_list struct and helpers pick 3029c3aca15 bundle-uri: create base key-value pair parsing pick a8b2de79ce8 bundle-uri: create "key=value" line parsing pick 92625a47673 bundle-uri: unit test "key=value" parsing pick a8616af4dc2 bundle-uri: limit recursion depth for bundle lists pick 9d6809a8d53 bundle-uri: parse bundle list in config format pick 287a732b54c bundle-uri: fetch a list of bundles exec git update-ref refs/heads/bundle-redo/list HEAD 287a732b54c4d95e7f410b3b36ef90d8a19cd346 pick b09f8226185 protocol v2: add server-side "bundle-uri" skeleton pick 520204dcd1c bundle-uri client: add minimal NOOP client pick 62e8b457b48 bundle-uri client: add "git ls-remote-bundle-uri" pick 00eae925043 bundle-uri: serve URI advertisement from bundle.* config pick 4277440a250 bundle-uri client: add boolean transfer.bundleURI setting pick caf4599a81d bundle-uri: allow relative URLs in bundle lists pick df255000b7e bundle-uri: download bundles from an advertised list pick d71beabf199 clone: unbundle the advertised bundles pick c9578391976 t5601: basic bundle URI tests # Ref refs/heads/bundle-redo/rfc-3 checked out at '/home/stolee/_git/git-bundles' exec git update-ref refs/heads/bundle-redo/advertise HEAD c9578391976ab9899c4e4f9b5fa2827650097305 The first two patches are helpers that are needed, but the full logic of the --update-refs option is introduced in patch 3. The config option is available in patch 4. Thanks, -Stolee Derrick Stolee (4): log-tree: create for_each_decoration() branch: add branch_checked_out() helper rebase: add --update-refs option rebase: add rebase.updateRefs config option Documentation/config/rebase.txt | 3 + Documentation/git-rebase.txt | 11 ++++ branch.c | 24 ++++--- branch.h | 8 +++ builtin/rebase.c | 10 +++ log-tree.c | 111 ++++++++++++++++++++++---------- log-tree.h | 4 ++ sequencer.c | 99 ++++++++++++++++++++++++++++ sequencer.h | 1 + t/t3404-rebase-interactive.sh | 34 ++++++++++ 10 files changed, 262 insertions(+), 43 deletions(-) base-commit: 2668e3608e47494f2f10ef2b6e69f08a84816bcb Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1247%2Fderrickstolee%2Frebase-keep-decorations-v1 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1247/derrickstolee/rebase-keep-decorations-v1 Pull-Request: https://github.com/gitgitgadget/git/pull/1247 -- gitgitgadget ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 1/4] log-tree: create for_each_decoration() 2022-06-03 13:37 [PATCH 0/4] rebase: update branches in multi-part topic Derrick Stolee via GitGitGadget @ 2022-06-03 13:37 ` Derrick Stolee via GitGitGadget 2022-06-03 17:39 ` Junio C Hamano 2022-06-03 13:37 ` [PATCH 2/4] branch: add branch_checked_out() helper Derrick Stolee via GitGitGadget ` (6 subsequent siblings) 7 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-03 13:37 UTC (permalink / raw) To: git; +Cc: gitster, johannes.schindelin, me, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> It can be helpful to iterate through all the decorations on a commit without necessarily writing them to a stream. Implement for_each_decoration() and reimplement format_decorations_extended() to use that iterator. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- log-tree.c | 111 ++++++++++++++++++++++++++++++++++++----------------- log-tree.h | 4 ++ 2 files changed, 80 insertions(+), 35 deletions(-) diff --git a/log-tree.c b/log-tree.c index d0ac0a6327a..b15a7c9db22 100644 --- a/log-tree.c +++ b/log-tree.c @@ -282,6 +282,54 @@ static void show_name(struct strbuf *sb, const struct name_decoration *decoratio strbuf_addstr(sb, decoration->name); } +struct format_decorations_context { + struct strbuf *sb; + int use_color; + const char *prefix; + const char *separator; + const char *suffix; + const char *color_commit; + const char *color_reset; + const struct name_decoration *current_and_HEAD; +}; + +static int append_decoration(const struct name_decoration *d, + void *data) +{ + struct format_decorations_context *ctx = data; + /* + * When both current and HEAD are there, only + * show HEAD->current where HEAD would have + * appeared, skipping the entry for current. + */ + if (d != ctx->current_and_HEAD) { + strbuf_addstr(ctx->sb, ctx->color_commit); + strbuf_addstr(ctx->sb, ctx->prefix); + strbuf_addstr(ctx->sb, ctx->color_reset); + strbuf_addstr(ctx->sb, decorate_get_color(ctx->use_color, d->type)); + if (d->type == DECORATION_REF_TAG) + strbuf_addstr(ctx->sb, "tag: "); + + show_name(ctx->sb, d); + + if (ctx->current_and_HEAD && + d->type == DECORATION_REF_HEAD) { + strbuf_addstr(ctx->sb, " -> "); + strbuf_addstr(ctx->sb, ctx->color_reset); + strbuf_addstr(ctx->sb, + decorate_get_color( + ctx->use_color, + ctx->current_and_HEAD->type)); + show_name(ctx->sb, ctx->current_and_HEAD); + } + strbuf_addstr(ctx->sb, ctx->color_reset); + + ctx->prefix = ctx->separator; + } + + return 0; +} + /* * The caller makes sure there is no funny color before calling. * format_decorations_extended makes sure the same after return. @@ -294,49 +342,42 @@ void format_decorations_extended(struct strbuf *sb, const char *suffix) { const struct name_decoration *decoration; - const struct name_decoration *current_and_HEAD; - const char *color_commit = - diff_get_color(use_color, DIFF_COMMIT); - const char *color_reset = - decorate_get_color(use_color, DECORATION_NONE); + struct format_decorations_context ctx = { + .sb = sb, + .use_color = use_color, + .prefix = prefix, + .separator = separator, + .suffix = suffix, + .color_commit = diff_get_color(use_color, DIFF_COMMIT), + .color_reset = decorate_get_color(use_color, DECORATION_NONE), + }; decoration = get_name_decoration(&commit->object); if (!decoration) return; - current_and_HEAD = current_pointed_by_HEAD(decoration); - while (decoration) { - /* - * When both current and HEAD are there, only - * show HEAD->current where HEAD would have - * appeared, skipping the entry for current. - */ - if (decoration != current_and_HEAD) { - strbuf_addstr(sb, color_commit); - strbuf_addstr(sb, prefix); - strbuf_addstr(sb, color_reset); - strbuf_addstr(sb, decorate_get_color(use_color, decoration->type)); - if (decoration->type == DECORATION_REF_TAG) - strbuf_addstr(sb, "tag: "); - - show_name(sb, decoration); - - if (current_and_HEAD && - decoration->type == DECORATION_REF_HEAD) { - strbuf_addstr(sb, " -> "); - strbuf_addstr(sb, color_reset); - strbuf_addstr(sb, decorate_get_color(use_color, current_and_HEAD->type)); - show_name(sb, current_and_HEAD); - } - strbuf_addstr(sb, color_reset); + ctx.current_and_HEAD = current_pointed_by_HEAD(decoration); - prefix = separator; - } + for_each_decoration(commit, append_decoration, &ctx); + + strbuf_addstr(sb, ctx.color_commit); + strbuf_addstr(sb, ctx.suffix); + strbuf_addstr(sb, ctx.color_reset); +} + +int for_each_decoration(const struct commit *c, decoration_fn fn, void *data) +{ + const struct name_decoration *decoration; + + decoration = get_name_decoration(&c->object); + while (decoration) { + int res; + if ((res = fn(decoration, data))) + return res; decoration = decoration->next; } - strbuf_addstr(sb, color_commit); - strbuf_addstr(sb, suffix); - strbuf_addstr(sb, color_reset); + + return 0; } void show_decorations(struct rev_info *opt, struct commit *commit) diff --git a/log-tree.h b/log-tree.h index e7e4641cf83..ea07da2625b 100644 --- a/log-tree.h +++ b/log-tree.h @@ -35,4 +35,8 @@ void fmt_output_commit(struct strbuf *, struct commit *, struct rev_info *); void fmt_output_subject(struct strbuf *, const char *subject, struct rev_info *); void fmt_output_email_subject(struct strbuf *, struct rev_info *); +typedef int decoration_fn(const struct name_decoration *d, + void *data); +int for_each_decoration(const struct commit *c, decoration_fn fn, void *data); + #endif -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH 1/4] log-tree: create for_each_decoration() 2022-06-03 13:37 ` [PATCH 1/4] log-tree: create for_each_decoration() Derrick Stolee via GitGitGadget @ 2022-06-03 17:39 ` Junio C Hamano 2022-06-03 17:58 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-06-03 17:39 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > const struct name_decoration *decoration; > + struct format_decorations_context ctx = { > + .sb = sb, > + .use_color = use_color, > + .prefix = prefix, > + .separator = separator, > + .suffix = suffix, > + .color_commit = diff_get_color(use_color, DIFF_COMMIT), > + .color_reset = decorate_get_color(use_color, DECORATION_NONE), > + }; > > decoration = get_name_decoration(&commit->object); > if (!decoration) > return; > > + ctx.current_and_HEAD = current_pointed_by_HEAD(decoration); > > + for_each_decoration(commit, append_decoration, &ctx); The function for_each_decoration() that does not take decoration but a commit felt iffy, especially because we already have called get_name_decoration() to obtain one for commit, and the API forces us to do that again at the beginning of for_each_decoration(). > + strbuf_addstr(sb, ctx.color_commit); > + strbuf_addstr(sb, ctx.suffix); > + strbuf_addstr(sb, ctx.color_reset); > +} > + > +int for_each_decoration(const struct commit *c, decoration_fn fn, void *data) > +{ > + const struct name_decoration *decoration; > + > + decoration = get_name_decoration(&c->object); > + while (decoration) { > + int res; > + if ((res = fn(decoration, data))) > + return res; > decoration = decoration->next; > } > + > + return 0; > } We'll know if this small waste is worth it when we see the new caller(s), I guess, but even if they start from commit, allowing them the same early return trick would require this piece of code: decoration = get_name_decoration(&commit->object); if (!decoration) return; in them, and even if they do not need it, it would be trivial for them to say for_each_decoration(get_name_decoration(&commit->object), do_whatever_they_need_to_do, &ctx); so, we may want to revisit this after we finish reading the series through. Making the iterator take name_decoration does not look too bad. > void show_decorations(struct rev_info *opt, struct commit *commit) > diff --git a/log-tree.h b/log-tree.h > index e7e4641cf83..ea07da2625b 100644 > --- a/log-tree.h > +++ b/log-tree.h > @@ -35,4 +35,8 @@ void fmt_output_commit(struct strbuf *, struct commit *, struct rev_info *); > void fmt_output_subject(struct strbuf *, const char *subject, struct rev_info *); > void fmt_output_email_subject(struct strbuf *, struct rev_info *); > > +typedef int decoration_fn(const struct name_decoration *d, > + void *data); > +int for_each_decoration(const struct commit *c, decoration_fn fn, void *data); > + > #endif ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 1/4] log-tree: create for_each_decoration() 2022-06-03 17:39 ` Junio C Hamano @ 2022-06-03 17:58 ` Derrick Stolee 2022-06-03 18:40 ` Junio C Hamano 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-06-03 17:58 UTC (permalink / raw) To: Junio C Hamano, Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me On 6/3/2022 1:39 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > >> const struct name_decoration *decoration; >> + struct format_decorations_context ctx = { >> + .sb = sb, >> + .use_color = use_color, >> + .prefix = prefix, >> + .separator = separator, >> + .suffix = suffix, >> + .color_commit = diff_get_color(use_color, DIFF_COMMIT), >> + .color_reset = decorate_get_color(use_color, DECORATION_NONE), >> + }; >> >> decoration = get_name_decoration(&commit->object); >> if (!decoration) >> return; >> >> + ctx.current_and_HEAD = current_pointed_by_HEAD(decoration); >> >> + for_each_decoration(commit, append_decoration, &ctx); > > The function for_each_decoration() that does not take decoration but > a commit felt iffy, especially because we already have called > get_name_decoration() to obtain one for commit, and the API forces > us to do that again at the beginning of for_each_decoration(). This is more an issue with this particular caller since it needs that current_pointed_by_HEAD() information. I had considered it as something to include in the prototype of decoration_fn so it could be incorporated into for_each_decoration(), but it ended up being overly clunky for this one consumer. I'm open to other ideas, though. >> + strbuf_addstr(sb, ctx.color_commit); >> + strbuf_addstr(sb, ctx.suffix); >> + strbuf_addstr(sb, ctx.color_reset); >> +} >> + >> +int for_each_decoration(const struct commit *c, decoration_fn fn, void *data) The goal of this method was to make it super simple to iterate without doing any prep work: just load the commit and go. >> +{ >> + const struct name_decoration *decoration; >> + >> + decoration = get_name_decoration(&c->object); >> + while (decoration) { >> + int res; >> + if ((res = fn(decoration, data))) >> + return res; >> decoration = decoration->next; >> } >> + >> + return 0; >> } > > We'll know if this small waste is worth it when we see the new > caller(s), I guess, but even if they start from commit, allowing > them the same early return trick would require this piece of code: > > decoration = get_name_decoration(&commit->object); > if (!decoration) > return; The iterator returns quickly because the while loop notices a NULL decoration. This use in format_decorations_extended() needs that check because of the current_pointed_by_HEAD(decoration) call based on the result being non-NULL. > in them, and even if they do not need it, it would be trivial for > them to say > > for_each_decoration(get_name_decoration(&commit->object), > do_whatever_they_need_to_do, &ctx); > > so, we may want to revisit this after we finish reading the series > through. Making the iterator take name_decoration does not look > too bad. You are right that this would work, but it's a bit messier. We could go this route to avoid the duplicate get_name_decoration() calls in format_decorations_extended(). Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 1/4] log-tree: create for_each_decoration() 2022-06-03 17:58 ` Derrick Stolee @ 2022-06-03 18:40 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-03 18:40 UTC (permalink / raw) To: Derrick Stolee Cc: Derrick Stolee via GitGitGadget, git, johannes.schindelin, me Derrick Stolee <derrickstolee@github.com> writes: >>> + for_each_decoration(commit, append_decoration, &ctx); >> >> The function for_each_decoration() that does not take decoration but >> a commit felt iffy,... >> ... >> for_each_decoration(get_name_decoration(&commit->object), >> do_whatever_they_need_to_do, &ctx); >> >> so, we may want to revisit this after we finish reading the series >> through. Making the iterator take name_decoration does not look >> too bad. > > You are right that this would work, but it's a bit messier. We > could go this route to avoid the duplicate get_name_decoration() > calls in format_decorations_extended(). Having a convenience function that takes a commit and let you walk over its decoration is OK, but calling it for_each_decoration() bothers me somewhat (which is where my comment started from after all). Hopefully it will stay unambiguous, and it will stay to be unnecessary to name it for_each_decoration_for_commit(), as I do not think we'd add "decoration" to things that are not commits (or if we added one, we probably will call it differently from "decoration"). So, OK. ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 2/4] branch: add branch_checked_out() helper 2022-06-03 13:37 [PATCH 0/4] rebase: update branches in multi-part topic Derrick Stolee via GitGitGadget 2022-06-03 13:37 ` [PATCH 1/4] log-tree: create for_each_decoration() Derrick Stolee via GitGitGadget @ 2022-06-03 13:37 ` Derrick Stolee via GitGitGadget 2022-06-03 18:31 ` Junio C Hamano 2022-06-03 13:37 ` [PATCH 3/4] rebase: add --update-refs option Derrick Stolee via GitGitGadget ` (5 subsequent siblings) 7 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-03 13:37 UTC (permalink / raw) To: git; +Cc: gitster, johannes.schindelin, me, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The validate_new_branchname() method contains a check to see if a branch is checked out in any non-bare worktree. This is intended to prevent a force push that will mess up an existing checkout. This helper is not suitable to performing just that check, because the method will die() when the branch is checked out instead of returning an error code. Extract branch_checked_out() and use it within validate_new_branchname(). Another caller will be added in a coming change. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- branch.c | 24 ++++++++++++++++-------- branch.h | 8 ++++++++ 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/branch.c b/branch.c index 2d6569b0c62..2e6419cdfa5 100644 --- a/branch.c +++ b/branch.c @@ -369,6 +369,19 @@ int validate_branchname(const char *name, struct strbuf *ref) return ref_exists(ref->buf); } +int branch_checked_out(const char *refname, char **path) +{ + struct worktree **worktrees = get_worktrees(); + const struct worktree *wt = find_shared_symref(worktrees, "HEAD", refname); + int result = wt && !wt->is_bare; + + if (result && path) + *path = xstrdup(wt->path); + + free_worktrees(worktrees); + return result; +} + /* * Check if a branch 'name' can be created as a new branch; die otherwise. * 'force' can be used when it is OK for the named branch already exists. @@ -377,9 +390,7 @@ int validate_branchname(const char *name, struct strbuf *ref) */ int validate_new_branchname(const char *name, struct strbuf *ref, int force) { - struct worktree **worktrees; - const struct worktree *wt; - + char *path; if (!validate_branchname(name, ref)) return 0; @@ -387,13 +398,10 @@ int validate_new_branchname(const char *name, struct strbuf *ref, int force) die(_("a branch named '%s' already exists"), ref->buf + strlen("refs/heads/")); - worktrees = get_worktrees(); - wt = find_shared_symref(worktrees, "HEAD", ref->buf); - if (wt && !wt->is_bare) + if (branch_checked_out(ref->buf, &path)) die(_("cannot force update the branch '%s' " "checked out at '%s'"), - ref->buf + strlen("refs/heads/"), wt->path); - free_worktrees(worktrees); + ref->buf + strlen("refs/heads/"), path); return 1; } diff --git a/branch.h b/branch.h index 560b6b96a8f..5ea93d217b1 100644 --- a/branch.h +++ b/branch.h @@ -101,6 +101,14 @@ void create_branches_recursively(struct repository *r, const char *name, const char *tracking_name, int force, int reflog, int quiet, enum branch_track track, int dry_run); + +/* + * Returns true if the branch at 'refname' is checked out at any + * non-bare worktree. The path of the worktree is stored in the + * given 'path', if provided. + */ +int branch_checked_out(const char *refname, char **path); + /* * Check if 'name' can be a valid name for a branch; die otherwise. * Return 1 if the named branch already exists; return 0 otherwise. -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH 2/4] branch: add branch_checked_out() helper 2022-06-03 13:37 ` [PATCH 2/4] branch: add branch_checked_out() helper Derrick Stolee via GitGitGadget @ 2022-06-03 18:31 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-03 18:31 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > +int branch_checked_out(const char *refname, char **path) > +{ > + struct worktree **worktrees = get_worktrees(); > + const struct worktree *wt = find_shared_symref(worktrees, "HEAD", refname); > + int result = wt && !wt->is_bare; > + > + if (result && path) > + *path = xstrdup(wt->path); > + > + free_worktrees(worktrees); > + return result; > +} Don't you plan to call this repeatedly from the for_each_deco iteration? I am wondering if it should take the result of get_worktrees() and reuse the result of get_worktrees(). There also was another topic that was triggered by find_shared_symref() being relatively heavy-weight, which suggests a more involved refactoring. I wonder if we rather want to rewrite find_shared_symref() *not* to take the target parameter at all, and instead introduce a new function that iterates over worktrees and report the branch that is checked out (or being operated on via rebase or bisect). Then we can - create a strset out of its result, i.e. set of branches that should not be touched; - iterate over refs that point into the history being rebased (using for_each_decoration()), and consult that strset to see if any of them is being rewritten. With the API of find_shared_symref(), we'd need to iterate over all worktrees for each decoration. With such a restructuring, we can iterate over all worktrees just once, and match the result with decoration, so the problem becomes O(N)+O(M) and not O(N*M) for number of worktrees N and number of decorations M. ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 3/4] rebase: add --update-refs option 2022-06-03 13:37 [PATCH 0/4] rebase: update branches in multi-part topic Derrick Stolee via GitGitGadget 2022-06-03 13:37 ` [PATCH 1/4] log-tree: create for_each_decoration() Derrick Stolee via GitGitGadget 2022-06-03 13:37 ` [PATCH 2/4] branch: add branch_checked_out() helper Derrick Stolee via GitGitGadget @ 2022-06-03 13:37 ` Derrick Stolee via GitGitGadget 2022-06-07 10:25 ` Phillip Wood 2022-06-03 13:37 ` [PATCH 4/4] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget ` (4 subsequent siblings) 7 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-03 13:37 UTC (permalink / raw) To: git; +Cc: gitster, johannes.schindelin, me, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> When working on a large feature, it can be helpful to break that feature into multiple smaller parts that become reviewed in sequence. During development or during review, a change to one part of the feature could affect multiple of these parts. An interactive rebase can help adjust the multi-part "story" of the branch. However, if there are branches tracking the different parts of the feature, then rebasing the entire list of commits can create commits not reachable from those "sub branches". It can take a manual step to update those branches. Add a new --update-refs option to 'git rebase -i' that adds 'git update-ref' exec steps to the todo file whenever a commit that is being rebased is decorated with that <ref>. This allows the user to rebase a long list of commits in a multi-part feature and keep all of their pointers to those parts. Use the new for_each_decoration() while iterating over the existing todo list. Be sure to iterate after any squashing or fixups are placed. Update the branch only after those squashes and fixups are complete. This allows a --fixup commit at the tip of the feature to apply correctly to the sub branch, even if it is fixing up the most-recent commit in that part. One potential problem here is that refs decorating commits that are already marked as "fixup!" or "squash!" will not be included in this list. Generally, the reordering of the "fixup!" and "squash!" is likely to change the relative order of these refs, so it is not recommended. The workflow here is intended to allow these kinds of commits at the tip of the rebased branch while the other sub branches come along for the ride without intervention. Be careful to not attempt updating any branch that is checked out. The most common example is the branch being rebased is checked out and decorates the tip commit. If the user is rebasing commits reachable from a different branch that is checked out in a different worktree, then they may be surprised to not see that ref update. However, it's probably best to not optimize for this scenario and do the safest thing that will result in a successful rebase. A comment is left in the TODO list that signals that these refs are currently checked out. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- Documentation/git-rebase.txt | 7 +++ builtin/rebase.c | 5 ++ sequencer.c | 99 +++++++++++++++++++++++++++++++++++ sequencer.h | 1 + t/t3404-rebase-interactive.sh | 29 ++++++++++ 5 files changed, 141 insertions(+) diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index 262fb01aec0..866554fc978 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -609,6 +609,13 @@ provided. Otherwise an explicit `--no-reschedule-failed-exec` at the start would be overridden by the presence of `rebase.rescheduleFailedExec=true` configuration. +--update-refs:: +--no-update-refs:: + Automatically force-update any branches that point to commits that + are being rebased. Any branches that are checked out in a worktree + or point to a `squash! ...` or `fixup! ...` commit are not updated + in this way. + INCOMPATIBLE OPTIONS -------------------- diff --git a/builtin/rebase.c b/builtin/rebase.c index 7ab50cda2ad..56d82a52106 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -102,6 +102,7 @@ struct rebase_options { int reschedule_failed_exec; int reapply_cherry_picks; int fork_point; + int update_refs; }; #define REBASE_OPTIONS_INIT { \ @@ -298,6 +299,7 @@ static int do_interactive_rebase(struct rebase_options *opts, unsigned flags) ret = complete_action(the_repository, &replay, flags, shortrevisions, opts->onto_name, opts->onto, &opts->orig_head, &commands, opts->autosquash, + opts->update_refs, &todo_list); } @@ -1124,6 +1126,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) OPT_BOOL(0, "autosquash", &options.autosquash, N_("move commits that begin with " "squash!/fixup! under -i")), + OPT_BOOL(0, "update-refs", &options.update_refs, + N_("update local refs that point to commits " + "that are being rebased")), { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"), N_("GPG-sign commits"), PARSE_OPT_OPTARG, NULL, (intptr_t) "" }, diff --git a/sequencer.c b/sequencer.c index 8c3ed3532ac..d6151af9849 100644 --- a/sequencer.c +++ b/sequencer.c @@ -35,6 +35,8 @@ #include "commit-reach.h" #include "rebase-interactive.h" #include "reset.h" +#include "branch.h" +#include "log-tree.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -5603,10 +5605,104 @@ static int skip_unnecessary_picks(struct repository *r, return 0; } +struct todo_add_branch_context { + struct todo_list new_list; + struct strbuf *buf; + struct commit *commit; +}; + +static int add_branch_for_decoration(const struct name_decoration *d, void *data) +{ + struct todo_add_branch_context *ctx = data; + size_t base_offset = ctx->buf->len; + int i = ctx->new_list.nr; + struct todo_item *item; + char *path; + + ALLOC_GROW(ctx->new_list.items, + ctx->new_list.nr + 1, + ctx->new_list.alloc); + item = &ctx->new_list.items[i]; + + /* If the branch is checked out, then leave a comment instead. */ + if (branch_checked_out(d->name, &path)) { + item->command = TODO_COMMENT; + strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", + d->name, path); + free(path); + } else { + item->command = TODO_EXEC; + strbuf_addf(ctx->buf, "git update-ref %s HEAD %s\n", + d->name, oid_to_hex(&ctx->commit->object.oid)); + } + + item->commit = NULL; + item->offset_in_buf = base_offset; + item->arg_offset = base_offset; + item->arg_len = ctx->buf->len - base_offset; + ctx->new_list.nr++; + + return 0; +} + +/* + * For each 'pick' command, find out if the commit has a decoration in + * refs/heads/. If so, then add a 'git branch -f' exec command after + * that 'pick' (plus any following 'squash' or 'fixup' commands). + */ +static int todo_list_add_branch_commands(struct todo_list *todo_list) +{ + int i; + static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP; + struct decoration_filter decoration_filter = { + .include_ref_pattern = &decorate_refs_include, + .exclude_ref_pattern = &decorate_refs_exclude, + .exclude_ref_config_pattern = &decorate_refs_exclude_config + }; + struct todo_add_branch_context ctx = { + .new_list = TODO_LIST_INIT, + .buf = &todo_list->buf, + }; + + string_list_append(&decorate_refs_include, "refs/heads/"); + load_ref_decorations(&decoration_filter, 0); + + for (i = 0; i < todo_list->nr; ) { + struct todo_item *item = &todo_list->items[i]; + + do { + /* insert ith item into new list */ + ALLOC_GROW(ctx.new_list.items, + ctx.new_list.nr + 1, + ctx.new_list.alloc); + + memcpy(&ctx.new_list.items[ctx.new_list.nr++], + &todo_list->items[i], + sizeof(struct todo_item)); + + i++; + } while (i < todo_list->nr && + todo_list->items[i].command != TODO_PICK); + + ctx.commit = item->commit; + for_each_decoration(item->commit, add_branch_for_decoration, &ctx); + } + + free(todo_list->items); + todo_list->items = ctx.new_list.items; + todo_list->nr = ctx.new_list.nr; + todo_list->alloc = ctx.new_list.alloc; + + return 0; +} + int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags, const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, + unsigned keep_decorations, struct todo_list *todo_list) { char shortonto[GIT_MAX_HEXSZ + 1]; @@ -5628,6 +5724,9 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla if (autosquash && todo_list_rearrange_squash(todo_list)) return -1; + if (keep_decorations && todo_list_add_branch_commands(todo_list)) + return -1; + if (commands->nr) todo_list_add_exec_commands(todo_list, commands); diff --git a/sequencer.h b/sequencer.h index da64473636b..1add3c7119f 100644 --- a/sequencer.h +++ b/sequencer.h @@ -166,6 +166,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, + unsigned keep_decorations, struct todo_list *todo_list); int todo_list_rearrange_squash(struct todo_list *todo_list); diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index f31afd4a547..38c7ef95e0e 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1743,6 +1743,35 @@ test_expect_success 'ORIG_HEAD is updated correctly' ' test_cmp_rev ORIG_HEAD test-orig-head@{1} ' +test_expect_success '--update-refs adds git branch commands' ' + git checkout -b update-refs no-conflict-branch && + test_commit extra fileX && + git commit --amend --fixup=L && + ( + set_cat_todo_editor && + git branch -f base A && + git branch -f first J && + git branch -f second J && + git branch -f third L && + + test_must_fail git rebase -i --autosquash --update-refs primary >todo && + + cat >expect <<-EOF && + pick $(git log -1 --format=%h J) J + exec git update-ref refs/heads/second HEAD $(git rev-parse J) + exec git update-ref refs/heads/first HEAD $(git rev-parse J) + pick $(git log -1 --format=%h K) K + pick $(git log -1 --format=%h L) L + fixup $(git log -1 --format=%h update-refs) fixup! L + exec git update-ref refs/heads/third HEAD $(git rev-parse L) + pick $(git log -1 --format=%h M) M + exec git update-ref refs/heads/no-conflict-branch HEAD $(git rev-parse M) + EOF + + test_cmp expect todo + ) +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH 3/4] rebase: add --update-refs option 2022-06-03 13:37 ` [PATCH 3/4] rebase: add --update-refs option Derrick Stolee via GitGitGadget @ 2022-06-07 10:25 ` Phillip Wood 0 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-06-07 10:25 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Derrick Stolee Hi Stolee Just a couple of minor comments. On 03/06/2022 14:37, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > [...] > diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt > index 262fb01aec0..866554fc978 100644 > --- a/Documentation/git-rebase.txt > +++ b/Documentation/git-rebase.txt > @@ -609,6 +609,13 @@ provided. Otherwise an explicit `--no-reschedule-failed-exec` at the > start would be overridden by the presence of > `rebase.rescheduleFailedExec=true` configuration. > > +--update-refs:: > +--no-update-refs:: > + Automatically force-update any branches that point to commits that > + are being rebased. Any branches that are checked out in a worktree > + or point to a `squash! ...` or `fixup! ...` commit are not updated > + in this way. > + > INCOMPATIBLE OPTIONS > -------------------- We should add --update-refs to the list of options that are incompatible with --apply. > diff --git a/sequencer.c b/sequencer.c > index 8c3ed3532ac..d6151af9849 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -35,6 +35,8 @@ > #include "commit-reach.h" > #include "rebase-interactive.h" > #include "reset.h" > +#include "branch.h" > +#include "log-tree.h" > > #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" > > @@ -5603,10 +5605,104 @@ static int skip_unnecessary_picks(struct repository *r, > return 0; > } > > +struct todo_add_branch_context { > + struct todo_list new_list; Rather than using a struct todo_list I think it would be simpler overall to add struct todo_list_item *item; size_t nr, alloc; instead, as I found it confusing that we were (correctly) using the strbuf of the old list when adding the update-ref line to the new list. > + struct strbuf *buf; > + struct commit *commit; > +}; > [...] > + for (i = 0; i < todo_list->nr; ) { > + struct todo_item *item = &todo_list->items[i]; > + > + do { > + /* insert ith item into new list */ > + ALLOC_GROW(ctx.new_list.items, > + ctx.new_list.nr + 1, > + ctx.new_list.alloc); > + > + memcpy(&ctx.new_list.items[ctx.new_list.nr++], > + &todo_list->items[i], > + sizeof(struct todo_item)); May be ctx.new_list.items[ctx.new_list.nr++] = todo_list->items[i++]; would be clearer Best Wishes Phillip ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 4/4] rebase: add rebase.updateRefs config option 2022-06-03 13:37 [PATCH 0/4] rebase: update branches in multi-part topic Derrick Stolee via GitGitGadget ` (2 preceding siblings ...) 2022-06-03 13:37 ` [PATCH 3/4] rebase: add --update-refs option Derrick Stolee via GitGitGadget @ 2022-06-03 13:37 ` Derrick Stolee via GitGitGadget 2022-06-03 16:56 ` [PATCH 0/4] rebase: update branches in multi-part topic Junio C Hamano ` (3 subsequent siblings) 7 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-03 13:37 UTC (permalink / raw) To: git; +Cc: gitster, johannes.schindelin, me, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The previous change added the --update-refs command-line option. For users who always want this mode, create the rebase.updateRefs config option which behaves the same way as rebase.autoSquash does with the --autosquash option. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- Documentation/config/rebase.txt | 3 +++ Documentation/git-rebase.txt | 4 ++++ builtin/rebase.c | 5 +++++ t/t3404-rebase-interactive.sh | 5 +++++ 4 files changed, 17 insertions(+) diff --git a/Documentation/config/rebase.txt b/Documentation/config/rebase.txt index 8c979cb20f2..f19bd0e0407 100644 --- a/Documentation/config/rebase.txt +++ b/Documentation/config/rebase.txt @@ -21,6 +21,9 @@ rebase.autoStash:: `--autostash` options of linkgit:git-rebase[1]. Defaults to false. +rebase.updateRefs:: + If set to true enable `--update-refs` option by default. + rebase.missingCommitsCheck:: If set to "warn", git rebase -i will print a warning if some commits are removed (e.g. a line was deleted), however the diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index 866554fc978..cb1b498bd99 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -615,6 +615,10 @@ start would be overridden by the presence of are being rebased. Any branches that are checked out in a worktree or point to a `squash! ...` or `fixup! ...` commit are not updated in this way. ++ +If the `--update-refs` option is enabled by default using the +configuration variable `rebase.updateRefs`, this option can be +used to override and disable this setting. INCOMPATIBLE OPTIONS -------------------- diff --git a/builtin/rebase.c b/builtin/rebase.c index 56d82a52106..8ebc98ea505 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -802,6 +802,11 @@ static int rebase_config(const char *var, const char *value, void *data) return 0; } + if (!strcmp(var, "rebase.updaterefs")) { + opts->update_refs = git_config_bool(var, value); + return 0; + } + if (!strcmp(var, "rebase.reschedulefailedexec")) { opts->reschedule_failed_exec = git_config_bool(var, value); return 0; diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 38c7ef95e0e..1fa9f78d40d 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1768,6 +1768,11 @@ test_expect_success '--update-refs adds git branch commands' ' exec git update-ref refs/heads/no-conflict-branch HEAD $(git rev-parse M) EOF + test_cmp expect todo && + + test_must_fail git -c rebase.autosquash=true \ + -c rebase.updaterefs=true \ + rebase -i primary >todo && test_cmp expect todo ) ' -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-03 13:37 [PATCH 0/4] rebase: update branches in multi-part topic Derrick Stolee via GitGitGadget ` (3 preceding siblings ...) 2022-06-03 13:37 ` [PATCH 4/4] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget @ 2022-06-03 16:56 ` Junio C Hamano 2022-06-03 18:27 ` Taylor Blau ` (2 subsequent siblings) 7 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-03 16:56 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > This is a feature I've wanted for quite a while. When working on the sparse > index topic, I created a long RFC that actually broke into three topics for > full review upstream. These topics were sequential, so any feedback on an > earlier one required updates to the later ones. I would work on the full > feature and use interactive rebase to update the full list of commits. > However, I would need to update the branches pointing to those sub-topics. > > This series adds a new --update-refs option to 'git rebase' (along with a > rebase.updateRefs config option) that adds 'git update-ref' commands into > the TODO list. This is powered by the commit decoration machinery. ;-) ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-03 13:37 [PATCH 0/4] rebase: update branches in multi-part topic Derrick Stolee via GitGitGadget ` (4 preceding siblings ...) 2022-06-03 16:56 ` [PATCH 0/4] rebase: update branches in multi-part topic Junio C Hamano @ 2022-06-03 18:27 ` Taylor Blau 2022-06-03 18:52 ` Junio C Hamano ` (2 more replies) 2022-06-07 6:25 ` Elijah Newren 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget 7 siblings, 3 replies; 144+ messages in thread From: Taylor Blau @ 2022-06-03 18:27 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, gitster, johannes.schindelin, Derrick Stolee On Fri, Jun 03, 2022 at 01:37:48PM +0000, Derrick Stolee via GitGitGadget wrote: > This is a feature I've wanted for quite a while. When working on the sparse > index topic, I created a long RFC that actually broke into three topics for > full review upstream. These topics were sequential, so any feedback on an > earlier one required updates to the later ones. I would work on the full > feature and use interactive rebase to update the full list of commits. > However, I would need to update the branches pointing to those sub-topics. This is really exciting. I'm glad that you're working on it, because I have also wanted this feature a handful of times over the years. This definitely would have come in handy when writing MIDX bitmaps, where I was often editing a local branch pointing at the final topic, and then trying to reconstruct all of the intermediate branches after each rebase. Not ever having to do that again sounds like a delight ;-). > pick 2d966282ff3 docs: document bundle URI standard > pick 31396e9171a remote-curl: add 'get' capability > pick 54c6ab70f67 bundle-uri: create basic file-copy logic > pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// > pick 6adaf842684 fetch: add --bundle-uri option > pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration > exec git update-ref refs/heads/bundle-redo/fetch HEAD 6c5840ed77e1bc41c1fe6fb7c894ceede1b8d730 But I wonder if we can or should delay these update-refs as long as possible. In particular: what happens if I get past this "exec" line (so that I've already updated my bundle-redo/fetch branch to point at the new thing), but decide at some later point to abort the rebase? I think users will expect us to restore bundle-redo/fetch to where it was before if we end up in that case. Recovering from it manually sounds like kind of a headache. What if instead we created labels here, and then delayed all ref updates to the end by replacing this with: label bundle-redo/fetch and then at the end of the todo list we'd add: exec git update-ref refs/heads/bundle-redo/fetch refs/rewritten/bundle-redo/fetch If we do all of those ref updates in a single transaction at the end, it should be much easier to roll back from if desired, and we'd avoid the aborted-rebase problem entirely. Thanks, Taylor ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-03 18:27 ` Taylor Blau @ 2022-06-03 18:52 ` Junio C Hamano 2022-06-03 19:59 ` Jeff Hostetler 2022-06-04 15:28 ` Phillip Wood 2 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-03 18:52 UTC (permalink / raw) To: Taylor Blau Cc: Derrick Stolee via GitGitGadget, git, johannes.schindelin, Derrick Stolee Taylor Blau <me@ttaylorr.com> writes: > But I wonder if we can or should delay these update-refs as long as > possible. In particular: what happens if I get past this "exec" line (so > that I've already updated my bundle-redo/fetch branch to point at the > new thing), but decide at some later point to abort the rebase? > > I think users will expect us to restore bundle-redo/fetch to where it > was before if we end up in that case. Recovering from it manually sounds > like kind of a headache. That is a very good safety and usability concern. I am glad somebody thought of it. > What if instead we created labels here, and then delayed all ref updates > to the end by replacing this with: > > label bundle-redo/fetch > > and then at the end of the todo list we'd add: > > exec git update-ref refs/heads/bundle-redo/fetch refs/rewritten/bundle-redo/fetch > > If we do all of those ref updates in a single transaction at the end, it > should be much easier to roll back from if desired, and we'd avoid the > aborted-rebase problem entirely. ;-) ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-03 18:27 ` Taylor Blau 2022-06-03 18:52 ` Junio C Hamano @ 2022-06-03 19:59 ` Jeff Hostetler 2022-06-03 20:03 ` Taylor Blau 2022-06-03 21:23 ` Junio C Hamano 2022-06-04 15:28 ` Phillip Wood 2 siblings, 2 replies; 144+ messages in thread From: Jeff Hostetler @ 2022-06-03 19:59 UTC (permalink / raw) To: Taylor Blau, Derrick Stolee via GitGitGadget Cc: git, gitster, johannes.schindelin, Derrick Stolee On 6/3/22 2:27 PM, Taylor Blau wrote: > On Fri, Jun 03, 2022 at 01:37:48PM +0000, Derrick Stolee via GitGitGadget wrote: >> This is a feature I've wanted for quite a while. When working on the sparse >> index topic, I created a long RFC that actually broke into three topics for >> full review upstream. These topics were sequential, so any feedback on an >> earlier one required updates to the later ones. I would work on the full >> feature and use interactive rebase to update the full list of commits. >> However, I would need to update the branches pointing to those sub-topics. > > This is really exciting. I'm glad that you're working on it, because I > have also wanted this feature a handful of times over the years. > > This definitely would have come in handy when writing MIDX bitmaps, > where I was often editing a local branch pointing at the final topic, > and then trying to reconstruct all of the intermediate branches after > each rebase. Not ever having to do that again sounds like a delight ;-). > >> pick 2d966282ff3 docs: document bundle URI standard >> pick 31396e9171a remote-curl: add 'get' capability >> pick 54c6ab70f67 bundle-uri: create basic file-copy logic >> pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// >> pick 6adaf842684 fetch: add --bundle-uri option >> pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration >> exec git update-ref refs/heads/bundle-redo/fetch HEAD 6c5840ed77e1bc41c1fe6fb7c894ceede1b8d730 > > But I wonder if we can or should delay these update-refs as long as > possible. In particular: what happens if I get past this "exec" line (so > that I've already updated my bundle-redo/fetch branch to point at the > new thing), but decide at some later point to abort the rebase? > > I think users will expect us to restore bundle-redo/fetch to where it > was before if we end up in that case. Recovering from it manually sounds > like kind of a headache. > > What if instead we created labels here, and then delayed all ref updates > to the end by replacing this with: > > label bundle-redo/fetch > > and then at the end of the todo list we'd add: > > exec git update-ref refs/heads/bundle-redo/fetch refs/rewritten/bundle-redo/fetch > > If we do all of those ref updates in a single transaction at the end, it > should be much easier to roll back from if desired, and we'd avoid the > aborted-rebase problem entirely. > > Thanks, > Taylor > I agree. I could have really used this while juggling all of the parts of FSMonitor recently. And yes, it should write the updates at the bottom in case of an abort. Should this take a branch pattern/regex to limit the set of branches that are updated (or offered to be updated)? For example, if I have an intermediate commit in the series that has 2 branch names pointing at it, do we want to offer to update both of them or only the one that matches some pattern related to the tip? Or is it sufficient to just enumerate them at the bottom of the todo list and let the user delete the lines they don't want? Should we actually do the update-ref's or should we write a script that lets the user do it later? The latter would let us also write out the commands to force update the remote refs if that would be helpful. Thanks, Jeff ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-03 19:59 ` Jeff Hostetler @ 2022-06-03 20:03 ` Taylor Blau 2022-06-03 21:23 ` Junio C Hamano 1 sibling, 0 replies; 144+ messages in thread From: Taylor Blau @ 2022-06-03 20:03 UTC (permalink / raw) To: Jeff Hostetler Cc: Taylor Blau, Derrick Stolee via GitGitGadget, git, gitster, johannes.schindelin, Derrick Stolee On Fri, Jun 03, 2022 at 03:59:00PM -0400, Jeff Hostetler wrote: > Should this take a branch pattern/regex to limit the set of branches > that are updated (or offered to be updated)? For example, if I have > an intermediate commit in the series that has 2 branch names pointing > at it, do we want to offer to update both of them or only the one > that matches some pattern related to the tip? Or is it sufficient to > just enumerate them at the bottom of the todo list and let the user > delete the lines they don't want? That could be a useful feature to layer on top. I don't think it's required here, though, since (as you note) users can remove lines from the update-ref invocation(s) at the bottom that they don't want to perform. > Should we actually do the update-ref's or should we write a script > that lets the user do it later? The latter would let us also write > out the commands to force update the remote refs if that would be > helpful. That seems like it would be outside the bounds of what `rebase` should provide, but others may feel differently. Thanks, Taylor ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-03 19:59 ` Jeff Hostetler 2022-06-03 20:03 ` Taylor Blau @ 2022-06-03 21:23 ` Junio C Hamano 1 sibling, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-03 21:23 UTC (permalink / raw) To: Jeff Hostetler Cc: Taylor Blau, Derrick Stolee via GitGitGadget, git, johannes.schindelin, Derrick Stolee Jeff Hostetler <git@jeffhostetler.com> writes: > I agree. I could have really used this while juggling all of the > parts of FSMonitor recently. And yes, it should write the updates > at the bottom in case of an abort. > > Should this take a branch pattern/regex to limit the set of branches > that are updated (or offered to be updated)? For example, if I have > an intermediate commit in the series that has 2 branch names pointing > at it, do we want to offer to update both of them or only the one > that matches some pattern related to the tip? Or is it sufficient to > just enumerate them at the bottom of the todo list and let the user > delete the lines they don't want? The latter sounds sufficient (starting from something simpler should work well in this case). > Should we actually do the update-ref's or should we write a script > that lets the user do it later? The latter would let us also write > out the commands to force update the remote refs if that would be > helpful. Aren't we writing "a script" already by implementing it as an additional "exec git update-ref" in the todo list already? I found that, combined with the idea to use the "label" Taylor mentioned, was the most brilliant part of this proposal. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-03 18:27 ` Taylor Blau 2022-06-03 18:52 ` Junio C Hamano 2022-06-03 19:59 ` Jeff Hostetler @ 2022-06-04 15:28 ` Phillip Wood 2022-06-06 15:12 ` Derrick Stolee 2022-06-06 16:36 ` Junio C Hamano 2 siblings, 2 replies; 144+ messages in thread From: Phillip Wood @ 2022-06-04 15:28 UTC (permalink / raw) To: Taylor Blau, Derrick Stolee via GitGitGadget Cc: git, gitster, johannes.schindelin, Derrick Stolee On 03/06/2022 19:27, Taylor Blau wrote: > On Fri, Jun 03, 2022 at 01:37:48PM +0000, Derrick Stolee via GitGitGadget wrote: >> This is a feature I've wanted for quite a while. When working on the sparse >> index topic, I created a long RFC that actually broke into three topics for >> full review upstream. These topics were sequential, so any feedback on an >> earlier one required updates to the later ones. I would work on the full >> feature and use interactive rebase to update the full list of commits. >> However, I would need to update the branches pointing to those sub-topics. > > This is really exciting. I'm glad that you're working on it, because I > have also wanted this feature a handful of times over the years. Yes, thank you Stolee. I agree this will be useful, but I wonder if users will be confused that --update-refs only updates the branch heads that happen to be in the todo list, rather than updating all the branches that contain a rewritten commit. I think the latter is something we should try to add in the future and so we should take care to design this topic so that is possible. > This definitely would have come in handy when writing MIDX bitmaps, > where I was often editing a local branch pointing at the final topic, > and then trying to reconstruct all of the intermediate branches after > each rebase. Not ever having to do that again sounds like a delight ;-). > >> pick 2d966282ff3 docs: document bundle URI standard >> pick 31396e9171a remote-curl: add 'get' capability >> pick 54c6ab70f67 bundle-uri: create basic file-copy logic >> pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// >> pick 6adaf842684 fetch: add --bundle-uri option >> pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration >> exec git update-ref refs/heads/bundle-redo/fetch HEAD 6c5840ed77e1bc41c1fe6fb7c894ceede1b8d730 > > But I wonder if we can or should delay these update-refs as long as > possible. In particular: what happens if I get past this "exec" line (so > that I've already updated my bundle-redo/fetch branch to point at the > new thing), but decide at some later point to abort the rebase? Absolutely! There is also the question of what to do if a user skips a commit that is a branch head. It is not obvious if they just want to drop that commit from the branch or if they want to skip updating the ref as well (I guess they can edit the todo list for the latter case). > I think users will expect us to restore bundle-redo/fetch to where it > was before if we end up in that case. Recovering from it manually sounds > like kind of a headache. > > What if instead we created labels here, and then delayed all ref updates > to the end by replacing this with: > > label bundle-redo/fetch Instead of using 'label' and 'exec' I'd prefer a new todo list command ('update-ref' or 'update-branch'?) used in place of 'label' that takes a branch name and updates the branch ref at the end of the rebase. That would make it easy to do all the updates in a single transaction as you suggested. Adding exec lines to do this makes the todo list messy and we have been trying to stop rebase forking all the time. Best Wishes Phillip > and then at the end of the todo list we'd add: > > exec git update-ref refs/heads/bundle-redo/fetch refs/rewritten/bundle-redo/fetch > > If we do all of those ref updates in a single transaction at the end, it > should be much easier to roll back from if desired, and we'd avoid the > aborted-rebase problem entirely. > Thanks, > Taylor ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-04 15:28 ` Phillip Wood @ 2022-06-06 15:12 ` Derrick Stolee 2022-06-07 10:11 ` Phillip Wood 2022-06-06 16:36 ` Junio C Hamano 1 sibling, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-06-06 15:12 UTC (permalink / raw) To: phillip.wood, Taylor Blau, Derrick Stolee via GitGitGadget Cc: git, gitster, johannes.schindelin On 6/4/2022 11:28 AM, Phillip Wood wrote: > On 03/06/2022 19:27, Taylor Blau wrote: >> On Fri, Jun 03, 2022 at 01:37:48PM +0000, Derrick Stolee via GitGitGadget wrote: >>> This is a feature I've wanted for quite a while. When working on the sparse >>> index topic, I created a long RFC that actually broke into three topics for >>> full review upstream. These topics were sequential, so any feedback on an >>> earlier one required updates to the later ones. I would work on the full >>> feature and use interactive rebase to update the full list of commits. >>> However, I would need to update the branches pointing to those sub-topics. >> >> This is really exciting. I'm glad that you're working on it, because I >> have also wanted this feature a handful of times over the years. > > Yes, thank you Stolee. I agree this will be useful, but I wonder if users > will be confused that --update-refs only updates the branch heads that happen > to be in the todo list, rather than updating all the branches that contain a > rewritten commit. I think the latter is something we should try to add in the > future and so we should take care to design this topic so that is possible. At the moment, the design adds a comment to the TODO list, showing which branches are not possible to move because they are checked out at another worktree (or is currently checked out and will be updated by the rebase itself). That seems like a good place to insert alternative logic in the future if we see a need for better behavior here. Unless: am I misunderstanding something about your concern here? Are you worried about refs outside of refs/heads/*? Are you concerned about it being _all_ refs/heads/* that we find? One potential way to extend this (in the future) is to make --update-refs take an optional string argument containing a refspec. This would replace the default refspec of refs/heads/*. >> This definitely would have come in handy when writing MIDX bitmaps, >> where I was often editing a local branch pointing at the final topic, >> and then trying to reconstruct all of the intermediate branches after >> each rebase. Not ever having to do that again sounds like a delight ;-). >> >>> pick 2d966282ff3 docs: document bundle URI standard >>> pick 31396e9171a remote-curl: add 'get' capability >>> pick 54c6ab70f67 bundle-uri: create basic file-copy logic >>> pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// >>> pick 6adaf842684 fetch: add --bundle-uri option >>> pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration >>> exec git update-ref refs/heads/bundle-redo/fetch HEAD 6c5840ed77e1bc41c1fe6fb7c894ceede1b8d730 >> >> But I wonder if we can or should delay these update-refs as long as >> possible. In particular: what happens if I get past this "exec" line (so >> that I've already updated my bundle-redo/fetch branch to point at the >> new thing), but decide at some later point to abort the rebase? > > Absolutely! There is also the question of what to do if a user skips a > commit that is a branch head. It is not obvious if they just want to drop> that commit from the branch or if they want to skip updating the ref as> well (I guess they can edit the todo list for the latter case). >> I think users will expect us to restore bundle-redo/fetch to where it >> was before if we end up in that case. Recovering from it manually sounds >> like kind of a headache. >> >> What if instead we created labels here, and then delayed all ref updates >> to the end by replacing this with: >> >> label bundle-redo/fetch > > Instead of using 'label' and 'exec' I'd prefer a new todo list command > ('update-ref' or 'update-branch'?) used in place of 'label' that takes a > branch name and updates the branch ref at the end of the rebase. That > would make it easy to do all the updates in a single transaction as you > suggested. Adding exec lines to do this makes the todo list messy and we > have been trying to stop rebase forking all the time. Thanks. Yes, a new 'update-refs' step at the end would be good to make it clear we want to rewrite the refs in one go without a possible interruption from the user. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-06 15:12 ` Derrick Stolee @ 2022-06-07 10:11 ` Phillip Wood 2022-06-07 19:39 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Phillip Wood @ 2022-06-07 10:11 UTC (permalink / raw) To: Derrick Stolee, phillip.wood, Taylor Blau, Derrick Stolee via GitGitGadget Cc: git, gitster, johannes.schindelin On 06/06/2022 16:12, Derrick Stolee wrote: > On 6/4/2022 11:28 AM, Phillip Wood wrote: >> On 03/06/2022 19:27, Taylor Blau wrote: >>> On Fri, Jun 03, 2022 at 01:37:48PM +0000, Derrick Stolee via GitGitGadget wrote: >>>> This is a feature I've wanted for quite a while. When working on the sparse >>>> index topic, I created a long RFC that actually broke into three topics for >>>> full review upstream. These topics were sequential, so any feedback on an >>>> earlier one required updates to the later ones. I would work on the full >>>> feature and use interactive rebase to update the full list of commits. >>>> However, I would need to update the branches pointing to those sub-topics. >>> >>> This is really exciting. I'm glad that you're working on it, because I >>> have also wanted this feature a handful of times over the years. >> >> Yes, thank you Stolee. I agree this will be useful, but I wonder if users >> will be confused that --update-refs only updates the branch heads that happen >> to be in the todo list, rather than updating all the branches that contain a >> rewritten commit. I think the latter is something we should try to add in the >> future and so we should take care to design this topic so that is possible. > > At the moment, the design adds a comment to the TODO list, showing which > branches are not possible to move because they are checked out at another > worktree (or is currently checked out and will be updated by the rebase > itself). That seems like a good place to insert alternative logic in the > future if we see a need for better behavior here. I think the question of whether to update branches that are checked out in another worktree is a question of whether it is less inconvenient to the user to skip the ref update and leave the user to manually update the branch or to update the ref and leave the worktree in a potentially awkward state if the user was half way through building a commit. The answer probably depends on the preferences of the user. I've been using a script that updates the refs for all the branches being rewritten for a while and have found it preferable to always update the ref rather than have to do it manually. My script also updates the worktree checkout to the new HEAD if there are no uncommitted changes which I have found very convenient. My preference is probably because I tend not to have uncommitted changes lying around in the worktrees whose branches get updated. > Unless: am I misunderstanding something about your concern here? Are you > worried about refs outside of refs/heads/*? Are you concerned about it > being _all_ refs/heads/* that we find? My concerns are primarily about being able to extend the --update-ref option in a backwards compatible way. I'd like to be able to add functionality to rebase all refs/heads/* that are descended from the commits that a simple rebase would rewrite. Say I want to edit $commit then I want the rebase to rewrite all the commits and update all the branches in in git rev-list $(git for-each-ref --contains $commit refs/heads/) ^$commit^@ Ideally we'd avoid adding a new commandline option when adding that. I think we could use an optional argument as you suggest below (though it would not be a refspec). > One potential way to extend this (in the future) is to make --update-refs > take an optional string argument containing a refspec. This would replace > the default refspec of refs/heads/*. > [...] >> Instead of using 'label' and 'exec' I'd prefer a new todo list command >> ('update-ref' or 'update-branch'?) used in place of 'label' that takes a >> branch name and updates the branch ref at the end of the rebase. That >> would make it easy to do all the updates in a single transaction as you >> suggested. Adding exec lines to do this makes the todo list messy and we >> have been trying to stop rebase forking all the time. > > Thanks. Yes, a new 'update-refs' step at the end would be good to make > it clear we want to rewrite the refs in one go without a possible > interruption from the user. That's great, it is a bit more work for you but I think it gives a much nicer UI. Best Wishes Phillip > Thanks, > -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-07 10:11 ` Phillip Wood @ 2022-06-07 19:39 ` Derrick Stolee 2022-06-08 16:03 ` Junio C Hamano 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-06-07 19:39 UTC (permalink / raw) To: phillip.wood, Taylor Blau, Derrick Stolee via GitGitGadget Cc: git, gitster, johannes.schindelin On 6/7/2022 6:11 AM, Phillip Wood wrote: > On 06/06/2022 16:12, Derrick Stolee wrote: >> On 6/4/2022 11:28 AM, Phillip Wood wrote: >>> On 03/06/2022 19:27, Taylor Blau wrote: >>>> On Fri, Jun 03, 2022 at 01:37:48PM +0000, Derrick Stolee via GitGitGadget wrote: >>>>> This is a feature I've wanted for quite a while. When working on the sparse >>>>> index topic, I created a long RFC that actually broke into three topics for >>>>> full review upstream. These topics were sequential, so any feedback on an >>>>> earlier one required updates to the later ones. I would work on the full >>>>> feature and use interactive rebase to update the full list of commits. >>>>> However, I would need to update the branches pointing to those sub-topics. >>>> >>>> This is really exciting. I'm glad that you're working on it, because I >>>> have also wanted this feature a handful of times over the years. >>> >>> Yes, thank you Stolee. I agree this will be useful, but I wonder if users >>> will be confused that --update-refs only updates the branch heads that happen >>> to be in the todo list, rather than updating all the branches that contain a >>> rewritten commit. I think the latter is something we should try to add in the >>> future and so we should take care to design this topic so that is possible. >> >> At the moment, the design adds a comment to the TODO list, showing which >> branches are not possible to move because they are checked out at another >> worktree (or is currently checked out and will be updated by the rebase >> itself). That seems like a good place to insert alternative logic in the >> future if we see a need for better behavior here. > > I think the question of whether to update branches that are checked out in > another worktree is a question of whether it is less inconvenient to the user > to skip the ref update and leave the user to manually update the branch or to> update the ref and leave the worktree in a potentially awkward state if the > user was half way through building a commit. The answer probably depends on > the preferences of the user. I think that their 'git status' will look strange no matter what: their working directory and index could look significantly different from what the branch at HEAD is reporting. For this situation, I would rather continue preventing these ref updates from underneath worktrees. > I've been using a script that updates the refs for all the branches being > rewritten for a while and have found it preferable to always update the ref > rather than have to do it manually. My script also updates the worktree > checkout to the new HEAD if there are no uncommitted changes which I have > found very convenient. My preference is probably because I tend not to have > uncommitted changes lying around in the worktrees whose branches get updated. Actually updating the worktree to match seems like an interesting twist, which we would want to consider if we go this route in the future. >> Unless: am I misunderstanding something about your concern here? Are you >> worried about refs outside of refs/heads/*? Are you concerned about it >> being _all_ refs/heads/* that we find? > > My concerns are primarily about being able to extend the --update-ref > option in a backwards compatible way. > > I'd like to be able to add functionality to rebase all refs/heads/* that > are descended from the commits that a simple rebase would rewrite. Say I want > to edit $commit then I want the rebase to rewrite all the commits and update > all the branches in in > > git rev-list $(git for-each-ref --contains $commit refs/heads/) ^$commit^@ > > Ideally we'd avoid adding a new commandline option when adding that. I think > we could use an optional argument as you suggest below (though it would not be > a refspec). This is sort of the opposite of what my series is doing: you want to find all refs that contain the current commits and make sure that they are _also_ rebased as we rebase the current set of commits. That seems like an interesting direction, with a very different set of complexities (ignoring other worktrees seems like much more of a blocker here). I would consider this to be _yet another mode_ that is separate from --update-refs, though it could share some underlying logic. It gets more complicated if there are merge commits involved (do we have a --rebase-merges option?). This makes me think of earlier discussions around a "git evolve" command (based on Mercurial's evolve command). The idea is to have a different workflow than I am presenting: instead of creating `fixup!` commits at the top of a multi-part topic, you could check out a branch on a middle part and work forward from there. Then, you could "rebase every branch with a rewritten commit" to get things back into a nice line. In conclusion: I don't think we should go down this rabbit hole right now. I think that --update-refs would be something we want to enable by default if we have such a mode, though. >> One potential way to extend this (in the future) is to make --update-refs >> take an optional string argument containing a refspec. This would replace >> the default refspec of refs/heads/*. >> [...] >>> Instead of using 'label' and 'exec' I'd prefer a new todo list command >>> ('update-ref' or 'update-branch'?) used in place of 'label' that takes a >>> branch name and updates the branch ref at the end of the rebase. That >>> would make it easy to do all the updates in a single transaction as you >>> suggested. Adding exec lines to do this makes the todo list messy and we >>> have been trying to stop rebase forking all the time. >> >> Thanks. Yes, a new 'update-refs' step at the end would be good to make >> it clear we want to rewrite the refs in one go without a possible >> interruption from the user. > > That's great, it is a bit more work for you but I think it gives a much > nicer UI. Thanks. After some hurdles on my end (and some additional complexities discovered in the process) I will have v2 ready soon. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-07 19:39 ` Derrick Stolee @ 2022-06-08 16:03 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-08 16:03 UTC (permalink / raw) To: Derrick Stolee Cc: phillip.wood, Taylor Blau, Derrick Stolee via GitGitGadget, git, johannes.schindelin Derrick Stolee <derrickstolee@github.com> writes: >> I think the question of whether to update branches that are checked out in >> another worktree is a question of whether it is less inconvenient to the user >> to skip the ref update and leave the user to manually update the branch or to >> update the ref and leave the worktree in a potentially awkward state if the >> user was half way through building a commit. The answer probably depends on >> the preferences of the user. To some degree, it might be true, but I think the recent thinking is that by default any refs being worked on by the user must be kept intact. "switch" does not let you check out a branch that is already checked out elsewhere, "fetch" does not let you overwrite the branch that is currently checked out without "--update-head-ok", etc. > I think that their 'git status' will look strange no matter what: their > working directory and index could look significantly different from what > the branch at HEAD is reporting. For this situation, I would rather continue > preventing these ref updates from underneath worktrees. > >> I've been using a script that updates the refs for all the branches being >> rewritten for a while and have found it preferable to always update the ref >> rather than have to do it manually. My script also updates the worktree >> checkout to the new HEAD if there are no uncommitted changes which I have >> found very convenient. My preference is probably because I tend not to have >> uncommitted changes lying around in the worktrees whose branches get updated. > > Actually updating the worktree to match seems like an interesting twist, which > we would want to consider if we go this route in the future. Usually I caution against adding "features" that can complicate the end-user experience like this, but in this case, the potential for extra conflicts coming from such updates to the working trees may not be too bad. It all depends on why the user has these intermediate branches and how they are used, but in order to see any conflicts happen after rebasing a branch here, before you start that rebase, you need to have in different working trees that had these intermediate branches checked out and had local modifications on top of them. I would say that you deserve it if conflicts resulting from such a set-up hurts you ;-) If you are using these intermediate branches so that you can work on steps in the middle of the larger whole, then the proposed "rebase --update-refs" is not for you. Having local modification in these separate working trees that check out the intermediate branches is OK, but once you create a commit on such a branch, the commit that is in the larger topic is no longer at the tip, so "rebase --update-refs" would not notice it anyway. Rather, if you have the intermediate branches, you'd probably want to work on each of them to make them stable, and rebase them on top of each other in an appropriate order. "rebase --update-refs" that runs on the larger topic does not have enough information to rebuild it with tips of intermediate branches that are updated elsewhere. Which means that anybody sane who uses "rebase --update-refs" would not modify these intermediate branches outside the context of the larger topic, and those who do not follow this rule can keep both halves of their history ;-) FWIW a complementary tool that would work in the other direction, to expect the user to have worked on smaller branches and rebuild the larger branch that contains them, is also missing from our tool set. Those who check out these smaller branches in separate working trees would want to use such a tool, not "rebase --update-ref". Thanks. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-04 15:28 ` Phillip Wood 2022-06-06 15:12 ` Derrick Stolee @ 2022-06-06 16:36 ` Junio C Hamano 1 sibling, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-06 16:36 UTC (permalink / raw) To: Phillip Wood Cc: Taylor Blau, Derrick Stolee via GitGitGadget, git, johannes.schindelin, Derrick Stolee Phillip Wood <phillip.wood123@gmail.com> writes: > Instead of using 'label' and 'exec' I'd prefer a new todo list command > ('update-ref' or 'update-branch'?) used in place of 'label' that takes > a branch name and updates the branch ref at the end of the > rebase. That would make it easy to do all the updates in a single > transaction as you suggested. Sounds like a good approach. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 0/4] rebase: update branches in multi-part topic 2022-06-03 13:37 [PATCH 0/4] rebase: update branches in multi-part topic Derrick Stolee via GitGitGadget ` (5 preceding siblings ...) 2022-06-03 18:27 ` Taylor Blau @ 2022-06-07 6:25 ` Elijah Newren 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget 7 siblings, 0 replies; 144+ messages in thread From: Elijah Newren @ 2022-06-07 6:25 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Derrick Stolee On Fri, Jun 3, 2022 at 11:01 AM Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com> wrote: > > This is a feature I've wanted for quite a while. When working on the sparse > index topic, I created a long RFC that actually broke into three topics for > full review upstream. These topics were sequential, so any feedback on an > earlier one required updates to the later ones. I would work on the full > feature and use interactive rebase to update the full list of commits. > However, I would need to update the branches pointing to those sub-topics. > > This series adds a new --update-refs option to 'git rebase' (along with a > rebase.updateRefs config option) that adds 'git update-ref' commands into > the TODO list. This is powered by the commit decoration machinery. > > As an example, here is my in-progress bundle URI RFC split into subtopics as > they appear during the TODO list of a git rebase -i --update-refs: > > pick 2d966282ff3 docs: document bundle URI standard > pick 31396e9171a remote-curl: add 'get' capability > pick 54c6ab70f67 bundle-uri: create basic file-copy logic > pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// > pick 6adaf842684 fetch: add --bundle-uri option > pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration > exec git update-ref refs/heads/bundle-redo/fetch HEAD 6c5840ed77e1bc41c1fe6fb7c894ceede1b8d730 > > pick 1e3f6546632 clone: add --bundle-uri option > pick 9e4a6fe9b68 clone: --bundle-uri cannot be combined with --depth > exec git update-ref refs/heads/bundle-redo/clone HEAD 9e4a6fe9b68a8455b427c9ac8cdbff30c96653b4 > > pick 5451cb6599c bundle-uri: create bundle_list struct and helpers > pick 3029c3aca15 bundle-uri: create base key-value pair parsing > pick a8b2de79ce8 bundle-uri: create "key=value" line parsing > pick 92625a47673 bundle-uri: unit test "key=value" parsing > pick a8616af4dc2 bundle-uri: limit recursion depth for bundle lists > pick 9d6809a8d53 bundle-uri: parse bundle list in config format > pick 287a732b54c bundle-uri: fetch a list of bundles > exec git update-ref refs/heads/bundle-redo/list HEAD 287a732b54c4d95e7f410b3b36ef90d8a19cd346 > > pick b09f8226185 protocol v2: add server-side "bundle-uri" skeleton > pick 520204dcd1c bundle-uri client: add minimal NOOP client > pick 62e8b457b48 bundle-uri client: add "git ls-remote-bundle-uri" > pick 00eae925043 bundle-uri: serve URI advertisement from bundle.* config > pick 4277440a250 bundle-uri client: add boolean transfer.bundleURI setting > pick caf4599a81d bundle-uri: allow relative URLs in bundle lists > pick df255000b7e bundle-uri: download bundles from an advertised list > pick d71beabf199 clone: unbundle the advertised bundles > pick c9578391976 t5601: basic bundle URI tests > # Ref refs/heads/bundle-redo/rfc-3 checked out at '/home/stolee/_git/git-bundles' > > exec git update-ref refs/heads/bundle-redo/advertise HEAD c9578391976ab9899c4e4f9b5fa2827650097305 > > > The first two patches are helpers that are needed, but the full logic of the > --update-refs option is introduced in patch 3. The config option is > available in patch 4. Very interesting; nice to see that others are thinking on a similar wavelength. I've got some related work in git-replay, and it heavily uses update-refs style handling. Not sure if there are any differences worth affecting your design or mine, but perhaps it's worth mentioning my current work and plans in git-replay related to the use of update-refs: Since git-replay doesn't touch the working tree or index, I had the natural question of what the output should be and whether ref-updates should automatically happen. I decided that like git-merge-tree, ref updates should not be automatic (and thus it behaves somewhat like a "dry run" version of rebase/cherry-pick). Since I also want to intrinsically handle replaying multiple branches simultaneously, the update-ref style output seemed like a natural fit to me (meaning users would pipe the output from git-reply to git-update-ref). Thus, one can do: $ git replay --onto main startpoint..mytopic and see git-update-ref style output: update refs/heads/mytopic ${NEW_mytopic_HASH} ${OLD_mytopic_HASH} where ${NEW_mytopic_HASH} is the commit at the tip of a chain of rebased mytopic commits. (Sidenote: There's the question of whether to update 'main' (i.e. cherry-picking) or 'mytopic' (i.e. rebasing); thus, users have to choose --advance vs. --onto to specify which of these should happen, except in cases where it can be implicitly determined.) Or, if you have several branches to update: $ git replay --onto main ^startpoint mytopic1 mytopic2 mytopic3 and see update refs/heads/mytopic1 ${NEW_mytopic1_HASH} ${OLD_mytopic1_HASH} update refs/heads/mytopic2 ${NEW_mytopic2_HASH} ${OLD_mytopic2_HASH} update refs/heads/mytopic3 ${NEW_mytopic3_HASH} ${OLD_mytopic3_HASH} (and yes, common commits from before will be replayed and then shared by the new branch versions.) If the topics are totally contained within each other, you can do this more simply as $ git replay --contained --onto main main..my_tip_topic and still see all the update commands: update refs/heads/mytopic1 ${NEW_mytopic1_HASH} ${OLD_mytopic1_HASH} update refs/heads/mytopic2 ${NEW_mytopic2_HASH} ${OLD_mytopic2_HASH} update refs/heads/mytopic3 ${NEW_mytopic3_HASH} ${OLD_mytopic3_HASH} update refs/heads/my_tip_topic ${NEW_my_tip_topic_HASH} ${OLD_my_tip_topic_HASH} The option to list several refs provides a natural way of allowing users to control what gets updated without updating all contained branches, though rebase's implicit ranges doesn't really have a nice analogue for you due to its implicit range handling. Right now, git-replay doesn't handle conflicts (it just die()s). That and a few other thing are still TODO. However, git-replay does already handle basic replaying of merges, including ones with evil changes (so long as none of the merges involved have conflicts). So, I'm hoping git-replay will also useful for stuff like: $ git replay --first-parent --interactive --onto next ^next special_topic seen where special_topic is one of the topics previously merged into seen. This last commit would allow seen to be rebuilt with some changes to special_topic. Basically, it makes whatever interactive changes are specified to special_topic followed by replaying all the merges in next..seen (including any necessary conflict resolutions or other semantic changes recorded in the merge commits being replayed), unless those . However, I do need to figure out some syntax for keeping special_topic from being transplanted onto next here... ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v2 0/7] rebase: update branches in multi-part topic 2022-06-03 13:37 [PATCH 0/4] rebase: update branches in multi-part topic Derrick Stolee via GitGitGadget ` (6 preceding siblings ...) 2022-06-07 6:25 ` Elijah Newren @ 2022-06-07 20:42 ` Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 1/7] log-tree: create for_each_decoration() Derrick Stolee via GitGitGadget ` (8 more replies) 7 siblings, 9 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee This is a feature I've wanted for quite a while. When working on the sparse index topic, I created a long RFC that actually broke into three topics for full review upstream. These topics were sequential, so any feedback on an earlier one required updates to the later ones. I would work on the full feature and use interactive rebase to update the full list of commits. However, I would need to update the branches pointing to those sub-topics. This series adds a new --update-refs option to 'git rebase' (along with a rebase.updateRefs config option) that adds 'label' and 'update-refs' commands into the TODO list. This is powered by the commit decoration machinery. As an example, here is my in-progress bundle URI RFC split into subtopics as they appear during the TODO list of a git rebase -i --update-refs: pick 2d966282ff3 docs: document bundle URI standard pick 31396e9171a remote-curl: add 'get' capability pick 54c6ab70f67 bundle-uri: create basic file-copy logic pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// pick 6adaf842684 fetch: add --bundle-uri option pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration label for-update-refs/refs/heads/bundle-redo/fetch pick 1e3f6546632 clone: add --bundle-uri option pick 9e4a6fe9b68 clone: --bundle-uri cannot be combined with --depth label for-update-refs/refs/heads/bundle-redo/clone pick 5451cb6599c bundle-uri: create bundle_list struct and helpers pick 3029c3aca15 bundle-uri: create base key-value pair parsing pick a8b2de79ce8 bundle-uri: create "key=value" line parsing pick 92625a47673 bundle-uri: unit test "key=value" parsing pick a8616af4dc2 bundle-uri: limit recursion depth for bundle lists pick 9d6809a8d53 bundle-uri: parse bundle list in config format pick 287a732b54c bundle-uri: fetch a list of bundles label for-update-refs/refs/heads/bundle-redo/list pick b09f8226185 protocol v2: add server-side "bundle-uri" skeleton pick 520204dcd1c bundle-uri client: add minimal NOOP client pick 62e8b457b48 bundle-uri client: add "git ls-remote-bundle-uri" pick 00eae925043 bundle-uri: serve URI advertisement from bundle.* config pick 4277440a250 bundle-uri client: add boolean transfer.bundleURI setting pick caf4599a81d bundle-uri: allow relative URLs in bundle lists pick df255000b7e bundle-uri: download bundles from an advertised list pick d71beabf199 clone: unbundle the advertised bundles pick c9578391976 t5601: basic bundle URI tests # Ref refs/heads/bundle-redo/rfc-3 checked out at '/home/stolee/_git/git-bundles' label for-update-refs/refs/heads/bundle-redo/advertise update-refs * Patches 1-2 are basic refactors that we will need for our implementation. * Patch 3 is a cleanup of an array with indexes given by the todo_command enum. * Patches 4-6 implement the feature incrementally. For instance, we can test the edits to the TODO file in patch 5, but we don't actually do anything when seeing the 'update-refs' command until patch 6. * Patch 7 adds the rebase.updateRefs config option similar to rebase.autoSquash. Updates in v2 ============= As recommended by the excellent feedback, I have removed the 'exec' commands in favor of the 'label' commands and a new 'update-refs' command at the very end. This way, there is only one step that updates all of the refs at the end instead of updating refs during the rebase. If a user runs 'git rebase --abort' in the middle, then their refs are still where they need to be. Based on some of the discussion, it seemed like one way to do this would be to have an 'update-ref ' command that would take the place of these 'label' commands. However, this would require two things that make it a bit awkward: 1. We would need to replicate the storage of those positions during the rebase. 'label' already does this pretty well. I've added the "for-update-refs/" label to help here. 2. If we want to close out all of the refs as the rebase is finishing, then that "step" becomes invisible to the user (and a bit more complicated to insert). Thus, the 'update-refs' step performs this action. If the user wants to do things after that step, then they can do so by editing the TODO list. Other updates: * The 'keep_decorations' parameter was renamed to 'update_refs'. * I added tests for --rebase-merges=rebase-cousins to show how these labels interact with other labels and merge commands. * I changed the order of the insertion of these update-refs labels to be before the fixups are rearranged. This fixes a bug where the tip commit is a fixup! so its decorations are never inspected (and they would be in the wrong place even if they were). The fixup! commands are properly inserted between a pick and its following label command. Tests demonstrate this is correct. * Numerous style choices are updated based on feedback. Thank you for all of the detailed review and ideas in this space. I appreciate any more ideas that can make this feature as effective as it can be. Thanks, -Stolee Derrick Stolee (7): log-tree: create for_each_decoration() branch: add branch_checked_out() helper sequencer: define array with enum values sequencer: add update-refs command rebase: add --update-refs option sequencer: implement 'update-refs' command rebase: add rebase.updateRefs config option Documentation/config/rebase.txt | 3 + Documentation/git-rebase.txt | 12 ++ branch.c | 24 ++-- branch.h | 8 ++ builtin/rebase.c | 10 ++ log-tree.c | 111 +++++++++++------ log-tree.h | 4 + sequencer.c | 206 +++++++++++++++++++++++++++++--- sequencer.h | 2 + t/t3404-rebase-interactive.sh | 110 +++++++++++++++++ 10 files changed, 432 insertions(+), 58 deletions(-) base-commit: 2668e3608e47494f2f10ef2b6e69f08a84816bcb Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1247%2Fderrickstolee%2Frebase-keep-decorations-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1247/derrickstolee/rebase-keep-decorations-v2 Pull-Request: https://github.com/gitgitgadget/git/pull/1247 Range-diff vs v1: 1: 4f9f3487641 = 1: 4f9f3487641 log-tree: create for_each_decoration() 2: 5f54766e103 = 2: 5f54766e103 branch: add branch_checked_out() helper -: ----------- > 3: 9f261c7df2c sequencer: define array with enum values -: ----------- > 4: 842b2186d25 sequencer: add update-refs command 3: 23fa6638864 ! 5: 0a4c110127b rebase: add --update-refs option @@ Commit message reachable from those "sub branches". It can take a manual step to update those branches. - Add a new --update-refs option to 'git rebase -i' that adds 'git - update-ref' exec steps to the todo file whenever a commit that is being - rebased is decorated with that <ref>. This allows the user to rebase a - long list of commits in a multi-part feature and keep all of their - pointers to those parts. + Add a new --update-refs option to 'git rebase -i' that adds 'label + for-update-refs/*' steps to the todo file whenever a commit that is + being rebased is decorated with that <ref>. At the very end, the + 'update-refs' step is added to update all of the branches referenced by + the 'label' steps. This allows the user to rebase a long list of commits + in a multi-part feature and keep all of their pointers to those parts. + + NOTE: This change only introduce the --update-refs option and implements + the changes to the todo file. It does _not_ yet implement the action + taken by the 'update-refs' todo step, which will be implemented and + tested in a later change. Use the new for_each_decoration() while iterating over the existing todo list. Be sure to iterate after any squashing or fixups are placed. @@ Documentation/git-rebase.txt: provided. Otherwise an explicit `--no-reschedule-f INCOMPATIBLE OPTIONS -------------------- +@@ Documentation/git-rebase.txt: are incompatible with the following options: + * --empty= + * --reapply-cherry-picks + * --edit-todo ++ * --update-refs + * --root when used in combination with --onto + + In addition, the following pairs of options are incompatible: ## builtin/rebase.c ## @@ builtin/rebase.c: struct rebase_options { @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, } +struct todo_add_branch_context { -+ struct todo_list new_list; ++ struct todo_item *items; ++ size_t items_nr; ++ size_t items_alloc; + struct strbuf *buf; + struct commit *commit; +}; @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, +{ + struct todo_add_branch_context *ctx = data; + size_t base_offset = ctx->buf->len; -+ int i = ctx->new_list.nr; + struct todo_item *item; + char *path; + -+ ALLOC_GROW(ctx->new_list.items, -+ ctx->new_list.nr + 1, -+ ctx->new_list.alloc); -+ item = &ctx->new_list.items[i]; ++ ALLOC_GROW(ctx->items, ++ ctx->items_nr + 1, ++ ctx->items_alloc); ++ item = &ctx->items[ctx->items_nr]; ++ memset(item, 0, sizeof(*item)); + + /* If the branch is checked out, then leave a comment instead. */ + if (branch_checked_out(d->name, &path)) { @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, + d->name, path); + free(path); + } else { -+ item->command = TODO_EXEC; -+ strbuf_addf(ctx->buf, "git update-ref %s HEAD %s\n", -+ d->name, oid_to_hex(&ctx->commit->object.oid)); ++ item->command = TODO_LABEL; ++ strbuf_addf(ctx->buf, "for-update-refs/%s\n", d->name); + } + -+ item->commit = NULL; + item->offset_in_buf = base_offset; + item->arg_offset = base_offset; + item->arg_len = ctx->buf->len - base_offset; -+ ctx->new_list.nr++; ++ ctx->items_nr++; + + return 0; +} + +/* + * For each 'pick' command, find out if the commit has a decoration in -+ * refs/heads/. If so, then add a 'git branch -f' exec command after -+ * that 'pick' (plus any following 'squash' or 'fixup' commands). ++ * refs/heads/. If so, then add a 'label for-update-refs/' command. + */ -+static int todo_list_add_branch_commands(struct todo_list *todo_list) ++static int todo_list_add_update_ref_commands(struct todo_list *todo_list) +{ + int i; + static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, + .exclude_ref_config_pattern = &decorate_refs_exclude_config + }; + struct todo_add_branch_context ctx = { -+ .new_list = TODO_LIST_INIT, + .buf = &todo_list->buf, + }; + ++ ctx.items_alloc = 2 * todo_list->nr + 1; ++ ALLOC_ARRAY(ctx.items, ctx.items_alloc); ++ + string_list_append(&decorate_refs_include, "refs/heads/"); + load_ref_decorations(&decoration_filter, 0); + + for (i = 0; i < todo_list->nr; ) { + struct todo_item *item = &todo_list->items[i]; + -+ do { -+ /* insert ith item into new list */ -+ ALLOC_GROW(ctx.new_list.items, -+ ctx.new_list.nr + 1, -+ ctx.new_list.alloc); ++ /* insert ith item into new list */ ++ ALLOC_GROW(ctx.items, ++ ctx.items_nr + 1, ++ ctx.items_alloc); + -+ memcpy(&ctx.new_list.items[ctx.new_list.nr++], -+ &todo_list->items[i], -+ sizeof(struct todo_item)); ++ ctx.items[ctx.items_nr++] = todo_list->items[i++]; + -+ i++; -+ } while (i < todo_list->nr && -+ todo_list->items[i].command != TODO_PICK); -+ -+ ctx.commit = item->commit; -+ for_each_decoration(item->commit, add_branch_for_decoration, &ctx); ++ if (item->commit) { ++ ctx.commit = item->commit; ++ for_each_decoration(item->commit, ++ add_branch_for_decoration, ++ &ctx); ++ } + } + ++ /* Add the "update-refs" step. */ ++ ALLOC_GROW(ctx.items, ++ ctx.items_nr + 1, ++ ctx.items_alloc); ++ memset(&ctx.items[ctx.items_nr], 0, sizeof(struct todo_item)); ++ ctx.items[ctx.items_nr].command = TODO_UPDATE_REFS; ++ ctx.items_nr++; ++ + free(todo_list->items); -+ todo_list->items = ctx.new_list.items; -+ todo_list->nr = ctx.new_list.nr; -+ todo_list->alloc = ctx.new_list.alloc; ++ todo_list->items = ctx.items; ++ todo_list->nr = ctx.items_nr; ++ todo_list->alloc = ctx.items_alloc; + + return 0; +} @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, -+ unsigned keep_decorations, ++ unsigned update_refs, struct todo_list *todo_list) { char shortonto[GIT_MAX_HEXSZ + 1]; @@ sequencer.c: int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla - if (autosquash && todo_list_rearrange_squash(todo_list)) - return -1; + item->arg_len = item->arg_offset = item->flags = item->offset_in_buf = 0; + } -+ if (keep_decorations && todo_list_add_branch_commands(todo_list)) ++ if (update_refs && todo_list_add_update_ref_commands(todo_list)) + return -1; + - if (commands->nr) - todo_list_add_exec_commands(todo_list, commands); + if (autosquash && todo_list_rearrange_squash(todo_list)) + return -1; ## sequencer.h ## @@ sequencer.h: int complete_action(struct repository *r, struct replay_opts *opts, const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, -+ unsigned keep_decorations, ++ unsigned update_refs, struct todo_list *todo_list); int todo_list_rearrange_squash(struct todo_list *todo_list); @@ t/t3404-rebase-interactive.sh: test_expect_success 'ORIG_HEAD is updated correct test_cmp_rev ORIG_HEAD test-orig-head@{1} ' -+test_expect_success '--update-refs adds git branch commands' ' ++test_expect_success '--update-refs adds label and update-ref commands' ' + git checkout -b update-refs no-conflict-branch && -+ test_commit extra fileX && -+ git commit --amend --fixup=L && ++ git branch -f base HEAD~4 && ++ git branch -f first HEAD~3 && ++ git branch -f second HEAD~3 && ++ git branch -f third HEAD~1 && ++ git commit --allow-empty --fixup=third && ++ git branch -f shared-tip && + ( + set_cat_todo_editor && -+ git branch -f base A && -+ git branch -f first J && -+ git branch -f second J && -+ git branch -f third L && -+ -+ test_must_fail git rebase -i --autosquash --update-refs primary >todo && + + cat >expect <<-EOF && + pick $(git log -1 --format=%h J) J -+ exec git update-ref refs/heads/second HEAD $(git rev-parse J) -+ exec git update-ref refs/heads/first HEAD $(git rev-parse J) ++ label for-update-refs/refs/heads/second ++ label for-update-refs/refs/heads/first + pick $(git log -1 --format=%h K) K + pick $(git log -1 --format=%h L) L -+ fixup $(git log -1 --format=%h update-refs) fixup! L -+ exec git update-ref refs/heads/third HEAD $(git rev-parse L) ++ fixup $(git log -1 --format=%h update-refs) fixup! L # empty ++ label for-update-refs/refs/heads/third + pick $(git log -1 --format=%h M) M -+ exec git update-ref refs/heads/no-conflict-branch HEAD $(git rev-parse M) ++ label for-update-refs/refs/heads/no-conflict-branch ++ label for-update-refs/refs/heads/shared-tip ++ update-refs + EOF + ++ test_must_fail git rebase -i --autosquash --update-refs primary >todo && ++ test_cmp expect todo ++ ) ++' ++ ++test_expect_success '--update-refs adds commands with --rebase-merges' ' ++ git checkout -b update-refs-with-merge no-conflict-branch && ++ git branch -f base HEAD~4 && ++ git branch -f first HEAD~3 && ++ git branch -f second HEAD~3 && ++ git branch -f third HEAD~1 && ++ git merge -m merge branch2 && ++ git branch -f merge-branch && ++ git commit --fixup=third --allow-empty && ++ ( ++ set_cat_todo_editor && ++ ++ cat >expect <<-EOF && ++ label onto ++ reset onto ++ pick $(git log -1 --format=%h branch2~1) F ++ pick $(git log -1 --format=%h branch2) I ++ label for-update-refs/refs/heads/branch2 ++ label merge ++ reset onto ++ pick $(git log -1 --format=%h refs/heads/second) J ++ label for-update-refs/refs/heads/second ++ label for-update-refs/refs/heads/first ++ pick $(git log -1 --format=%h refs/heads/third~1) K ++ pick $(git log -1 --format=%h refs/heads/third) L ++ fixup $(git log -1 --format=%h update-refs-with-merge) fixup! L # empty ++ label for-update-refs/refs/heads/third ++ pick $(git log -1 --format=%h HEAD~2) M ++ label for-update-refs/refs/heads/no-conflict-branch ++ merge -C $(git log -1 --format=%h HEAD~1) merge # merge ++ label for-update-refs/refs/heads/merge-branch ++ update-refs ++ EOF ++ ++ test_must_fail git rebase -i --autosquash \ ++ --rebase-merges=rebase-cousins \ ++ --update-refs primary >todo && ++ + test_cmp expect todo + ) +' -: ----------- > 6: 68f8e51b19c sequencer: implement 'update-refs' command 4: b99c5bf34ef ! 7: 3d7d3f656b4 rebase: add rebase.updateRefs config option @@ builtin/rebase.c: static int rebase_config(const char *var, const char *value, v return 0; ## t/t3404-rebase-interactive.sh ## -@@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds git branch commands' ' - exec git update-ref refs/heads/no-conflict-branch HEAD $(git rev-parse M) +@@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds label and update-ref commands' ' EOF + test_must_fail git rebase -i --autosquash --update-refs primary >todo && + test_cmp expect todo && + + test_must_fail git -c rebase.autosquash=true \ + -c rebase.updaterefs=true \ + rebase -i primary >todo && ++ + test_cmp expect todo + ) + ' +@@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds commands with --rebase-merges' ' + --rebase-merges=rebase-cousins \ + --update-refs primary >todo && + ++ test_cmp expect todo && ++ ++ test_must_fail git -c rebase.autosquash=true \ ++ -c rebase.updaterefs=true \ ++ rebase -i \ ++ --rebase-merges=rebase-cousins \ ++ primary >todo && ++ test_cmp expect todo ) ' -- gitgitgadget ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v2 1/7] log-tree: create for_each_decoration() 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 ` Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 2/7] branch: add branch_checked_out() helper Derrick Stolee via GitGitGadget ` (7 subsequent siblings) 8 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> It can be helpful to iterate through all the decorations on a commit without necessarily writing them to a stream. Implement for_each_decoration() and reimplement format_decorations_extended() to use that iterator. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- log-tree.c | 111 ++++++++++++++++++++++++++++++++++++----------------- log-tree.h | 4 ++ 2 files changed, 80 insertions(+), 35 deletions(-) diff --git a/log-tree.c b/log-tree.c index d0ac0a6327a..b15a7c9db22 100644 --- a/log-tree.c +++ b/log-tree.c @@ -282,6 +282,54 @@ static void show_name(struct strbuf *sb, const struct name_decoration *decoratio strbuf_addstr(sb, decoration->name); } +struct format_decorations_context { + struct strbuf *sb; + int use_color; + const char *prefix; + const char *separator; + const char *suffix; + const char *color_commit; + const char *color_reset; + const struct name_decoration *current_and_HEAD; +}; + +static int append_decoration(const struct name_decoration *d, + void *data) +{ + struct format_decorations_context *ctx = data; + /* + * When both current and HEAD are there, only + * show HEAD->current where HEAD would have + * appeared, skipping the entry for current. + */ + if (d != ctx->current_and_HEAD) { + strbuf_addstr(ctx->sb, ctx->color_commit); + strbuf_addstr(ctx->sb, ctx->prefix); + strbuf_addstr(ctx->sb, ctx->color_reset); + strbuf_addstr(ctx->sb, decorate_get_color(ctx->use_color, d->type)); + if (d->type == DECORATION_REF_TAG) + strbuf_addstr(ctx->sb, "tag: "); + + show_name(ctx->sb, d); + + if (ctx->current_and_HEAD && + d->type == DECORATION_REF_HEAD) { + strbuf_addstr(ctx->sb, " -> "); + strbuf_addstr(ctx->sb, ctx->color_reset); + strbuf_addstr(ctx->sb, + decorate_get_color( + ctx->use_color, + ctx->current_and_HEAD->type)); + show_name(ctx->sb, ctx->current_and_HEAD); + } + strbuf_addstr(ctx->sb, ctx->color_reset); + + ctx->prefix = ctx->separator; + } + + return 0; +} + /* * The caller makes sure there is no funny color before calling. * format_decorations_extended makes sure the same after return. @@ -294,49 +342,42 @@ void format_decorations_extended(struct strbuf *sb, const char *suffix) { const struct name_decoration *decoration; - const struct name_decoration *current_and_HEAD; - const char *color_commit = - diff_get_color(use_color, DIFF_COMMIT); - const char *color_reset = - decorate_get_color(use_color, DECORATION_NONE); + struct format_decorations_context ctx = { + .sb = sb, + .use_color = use_color, + .prefix = prefix, + .separator = separator, + .suffix = suffix, + .color_commit = diff_get_color(use_color, DIFF_COMMIT), + .color_reset = decorate_get_color(use_color, DECORATION_NONE), + }; decoration = get_name_decoration(&commit->object); if (!decoration) return; - current_and_HEAD = current_pointed_by_HEAD(decoration); - while (decoration) { - /* - * When both current and HEAD are there, only - * show HEAD->current where HEAD would have - * appeared, skipping the entry for current. - */ - if (decoration != current_and_HEAD) { - strbuf_addstr(sb, color_commit); - strbuf_addstr(sb, prefix); - strbuf_addstr(sb, color_reset); - strbuf_addstr(sb, decorate_get_color(use_color, decoration->type)); - if (decoration->type == DECORATION_REF_TAG) - strbuf_addstr(sb, "tag: "); - - show_name(sb, decoration); - - if (current_and_HEAD && - decoration->type == DECORATION_REF_HEAD) { - strbuf_addstr(sb, " -> "); - strbuf_addstr(sb, color_reset); - strbuf_addstr(sb, decorate_get_color(use_color, current_and_HEAD->type)); - show_name(sb, current_and_HEAD); - } - strbuf_addstr(sb, color_reset); + ctx.current_and_HEAD = current_pointed_by_HEAD(decoration); - prefix = separator; - } + for_each_decoration(commit, append_decoration, &ctx); + + strbuf_addstr(sb, ctx.color_commit); + strbuf_addstr(sb, ctx.suffix); + strbuf_addstr(sb, ctx.color_reset); +} + +int for_each_decoration(const struct commit *c, decoration_fn fn, void *data) +{ + const struct name_decoration *decoration; + + decoration = get_name_decoration(&c->object); + while (decoration) { + int res; + if ((res = fn(decoration, data))) + return res; decoration = decoration->next; } - strbuf_addstr(sb, color_commit); - strbuf_addstr(sb, suffix); - strbuf_addstr(sb, color_reset); + + return 0; } void show_decorations(struct rev_info *opt, struct commit *commit) diff --git a/log-tree.h b/log-tree.h index e7e4641cf83..ea07da2625b 100644 --- a/log-tree.h +++ b/log-tree.h @@ -35,4 +35,8 @@ void fmt_output_commit(struct strbuf *, struct commit *, struct rev_info *); void fmt_output_subject(struct strbuf *, const char *subject, struct rev_info *); void fmt_output_email_subject(struct strbuf *, struct rev_info *); +typedef int decoration_fn(const struct name_decoration *d, + void *data); +int for_each_decoration(const struct commit *c, decoration_fn fn, void *data); + #endif -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v2 2/7] branch: add branch_checked_out() helper 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 1/7] log-tree: create for_each_decoration() Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 ` Derrick Stolee via GitGitGadget 2022-06-07 22:09 ` Junio C Hamano 2022-06-07 20:42 ` [PATCH v2 3/7] sequencer: define array with enum values Derrick Stolee via GitGitGadget ` (6 subsequent siblings) 8 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The validate_new_branchname() method contains a check to see if a branch is checked out in any non-bare worktree. This is intended to prevent a force push that will mess up an existing checkout. This helper is not suitable to performing just that check, because the method will die() when the branch is checked out instead of returning an error code. Extract branch_checked_out() and use it within validate_new_branchname(). Another caller will be added in a coming change. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- branch.c | 24 ++++++++++++++++-------- branch.h | 8 ++++++++ 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/branch.c b/branch.c index 2d6569b0c62..2e6419cdfa5 100644 --- a/branch.c +++ b/branch.c @@ -369,6 +369,19 @@ int validate_branchname(const char *name, struct strbuf *ref) return ref_exists(ref->buf); } +int branch_checked_out(const char *refname, char **path) +{ + struct worktree **worktrees = get_worktrees(); + const struct worktree *wt = find_shared_symref(worktrees, "HEAD", refname); + int result = wt && !wt->is_bare; + + if (result && path) + *path = xstrdup(wt->path); + + free_worktrees(worktrees); + return result; +} + /* * Check if a branch 'name' can be created as a new branch; die otherwise. * 'force' can be used when it is OK for the named branch already exists. @@ -377,9 +390,7 @@ int validate_branchname(const char *name, struct strbuf *ref) */ int validate_new_branchname(const char *name, struct strbuf *ref, int force) { - struct worktree **worktrees; - const struct worktree *wt; - + char *path; if (!validate_branchname(name, ref)) return 0; @@ -387,13 +398,10 @@ int validate_new_branchname(const char *name, struct strbuf *ref, int force) die(_("a branch named '%s' already exists"), ref->buf + strlen("refs/heads/")); - worktrees = get_worktrees(); - wt = find_shared_symref(worktrees, "HEAD", ref->buf); - if (wt && !wt->is_bare) + if (branch_checked_out(ref->buf, &path)) die(_("cannot force update the branch '%s' " "checked out at '%s'"), - ref->buf + strlen("refs/heads/"), wt->path); - free_worktrees(worktrees); + ref->buf + strlen("refs/heads/"), path); return 1; } diff --git a/branch.h b/branch.h index 560b6b96a8f..5ea93d217b1 100644 --- a/branch.h +++ b/branch.h @@ -101,6 +101,14 @@ void create_branches_recursively(struct repository *r, const char *name, const char *tracking_name, int force, int reflog, int quiet, enum branch_track track, int dry_run); + +/* + * Returns true if the branch at 'refname' is checked out at any + * non-bare worktree. The path of the worktree is stored in the + * given 'path', if provided. + */ +int branch_checked_out(const char *refname, char **path); + /* * Check if 'name' can be a valid name for a branch; die otherwise. * Return 1 if the named branch already exists; return 0 otherwise. -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v2 2/7] branch: add branch_checked_out() helper 2022-06-07 20:42 ` [PATCH v2 2/7] branch: add branch_checked_out() helper Derrick Stolee via GitGitGadget @ 2022-06-07 22:09 ` Junio C Hamano 2022-06-08 2:14 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-06-07 22:09 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > +int branch_checked_out(const char *refname, char **path) > +{ > + struct worktree **worktrees = get_worktrees(); > + const struct worktree *wt = find_shared_symref(worktrees, "HEAD", refname); > + int result = wt && !wt->is_bare; > + > + if (result && path) > + *path = xstrdup(wt->path); > + > + free_worktrees(worktrees); > + return result; > +} Don't you plan to call this repeatedly from the for_each_deco iteration? I am wondering if it should take the result of get_worktrees() and reuse the result of get_worktrees(), instead of enumerating the list of worktrees and freeing for each of the branches you need to inspect. There also was another topic https://lore.kernel.org/git/pull.1266.v2.git.git.1652690501963.gitgitgadget@gmail.com/ that was triggered by find_shared_symref() being relatively heavy-weight, which suggests a more involved refactoring. I wonder if we rather want to rewrite find_shared_symref() *not* to take the target parameter at all, and instead introduce a new function that takes a worktree, and report the branch that is checked out (or being operated on via rebase or bisect). Then we can - create a strset out of its result, i.e. set of branches that should not be touched; - iterate over refs that point into the history being rebased (using for_each_decoration()), and consult that strset to see if any of them is being rewritten. With the API of find_shared_symref(), we'd need to iterate over all worktrees for each decoration. With such a restructuring, we can iterate over all worktrees just once, and match the result with decoration, so the problem becomes O(N)+O(M) and not O(N*M) for number of worktrees N and number of decorations M. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v2 2/7] branch: add branch_checked_out() helper 2022-06-07 22:09 ` Junio C Hamano @ 2022-06-08 2:14 ` Derrick Stolee 2022-06-08 2:43 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-06-08 2:14 UTC (permalink / raw) To: Junio C Hamano, Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 6/7/2022 6:09 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > >> +int branch_checked_out(const char *refname, char **path) >> +{ >> + struct worktree **worktrees = get_worktrees(); >> + const struct worktree *wt = find_shared_symref(worktrees, "HEAD", refname); >> + int result = wt && !wt->is_bare; >> + >> + if (result && path) >> + *path = xstrdup(wt->path); >> + >> + free_worktrees(worktrees); >> + return result; >> +} > > Don't you plan to call this repeatedly from the for_each_deco > iteration? I am wondering if it should take the result of > get_worktrees() and reuse the result of get_worktrees(), instead of > enumerating the list of worktrees and freeing for each of the > branches you need to inspect. Sorry, you had mentioned this in v1 and I just forgot to fix this. > There also was another topic > > https://lore.kernel.org/git/pull.1266.v2.git.git.1652690501963.gitgitgadget@gmail.com/ > > that was triggered by find_shared_symref() being relatively > heavy-weight, which suggests a more involved refactoring. > > I wonder if we rather want to rewrite find_shared_symref() *not* to > take the target parameter at all, and instead introduce a new > function that takes a worktree, and report the branch that is > checked out (or being operated on via rebase or bisect). Then we > can > > - create a strset out of its result, i.e. set of branches that > should not be touched; > > - iterate over refs that point into the history being rebased > (using for_each_decoration()), and consult that strset to see if > any of them is being rewritten. > > With the API of find_shared_symref(), we'd need to iterate over all > worktrees for each decoration. With such a restructuring, we can > iterate over all worktrees just once, and match the result with > decoration, so the problem becomes O(N)+O(M) and not O(N*M) for > number of worktrees N and number of decorations M. Yes, that's a good plan. I'll take a look. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v2 2/7] branch: add branch_checked_out() helper 2022-06-08 2:14 ` Derrick Stolee @ 2022-06-08 2:43 ` Derrick Stolee 2022-06-08 4:52 ` Junio C Hamano 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-06-08 2:43 UTC (permalink / raw) To: Junio C Hamano, Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 6/7/2022 10:14 PM, Derrick Stolee wrote: > On 6/7/2022 6:09 PM, Junio C Hamano wrote: >> >> I wonder if we rather want to rewrite find_shared_symref() *not* to >> take the target parameter at all, and instead introduce a new >> function that takes a worktree, and report the branch that is >> checked out (or being operated on via rebase or bisect). Then we >> can >> >> - create a strset out of its result, i.e. set of branches that >> should not be touched; >> >> - iterate over refs that point into the history being rebased >> (using for_each_decoration()), and consult that strset to see if >> any of them is being rewritten. >> >> With the API of find_shared_symref(), we'd need to iterate over all >> worktrees for each decoration. With such a restructuring, we can >> iterate over all worktrees just once, and match the result with >> decoration, so the problem becomes O(N)+O(M) and not O(N*M) for >> number of worktrees N and number of decorations M. > > Yes, that's a good plan. I'll take a look. Here is a fixup for this patch that should work. I'll put it in v3. diff --git a/branch.c b/branch.c index 2e6419cdfa5..514212f5619 100644 --- a/branch.c +++ b/branch.c @@ -10,6 +10,7 @@ #include "worktree.h" #include "submodule-config.h" #include "run-command.h" +#include "strmap.h" struct tracking { struct refspec_item spec; @@ -369,17 +370,44 @@ int validate_branchname(const char *name, struct strbuf *ref) return ref_exists(ref->buf); } -int branch_checked_out(const char *refname, char **path) +static int initialized_checked_out_branches; +static struct strmap current_checked_out_branches = STRMAP_INIT; + +static void prepare_checked_out_branches(void) { - struct worktree **worktrees = get_worktrees(); - const struct worktree *wt = find_shared_symref(worktrees, "HEAD", refname); - int result = wt && !wt->is_bare; + int i = 0; + struct worktree **worktrees; + + if (initialized_checked_out_branches) + return; + initialized_checked_out_branches = 1; + + worktrees = get_worktrees(); + + while (worktrees[i]) { + struct worktree *wt = worktrees[i]; - if (result && path) - *path = xstrdup(wt->path); + if (!wt->is_bare && wt->head_ref) + strmap_put(¤t_checked_out_branches, + wt->head_ref, + wt->path); + + i++; + } free_worktrees(worktrees); - return result; +} + +int branch_checked_out(const char *refname, char **path) +{ + const char *path_in_set; + prepare_checked_out_branches(); + + path_in_set = strmap_get(¤t_checked_out_branches, refname); + if (path_in_set && path) + *path = xstrdup(path_in_set); + + return !!path_in_set; } /* >> There also was another topic >> >> https://lore.kernel.org/git/pull.1266.v2.git.git.1652690501963.gitgitgadget@gmail.com/ >> >> that was triggered by find_shared_symref() being relatively >> heavy-weight, which suggests a more involved refactoring. The patch there was this: diff --git a/builtin/fetch.c b/builtin/fetch.c index e3791f09ed5..eeee5ac8f15 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1440,6 +1440,7 @@ static void check_not_current_branch(struct ref *ref_map, const struct worktree *wt; for (; ref_map; ref_map = ref_map->next) if (ref_map->peer_ref && + starts_with(ref_map->peer_ref->name, "refs/heads/") && (wt = find_shared_symref(worktrees, "HEAD", ref_map->peer_ref->name)) && !wt->is_bare) And there is another use of find_shared_symref() in the same file, allowing us to do the following: diff --git a/builtin/fetch.c b/builtin/fetch.c index ac29c2b1ae3..3933c482839 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -885,7 +885,7 @@ static int update_local_ref(struct ref *ref, struct worktree **worktrees) { struct commit *current = NULL, *updated; - const struct worktree *wt; + char *path = NULL; const char *pretty_ref = prettify_refname(ref->name); int fast_forward = 0; @@ -900,17 +900,17 @@ static int update_local_ref(struct ref *ref, } if (!update_head_ok && - (wt = find_shared_symref(worktrees, "HEAD", ref->name)) && - !wt->is_bare && !is_null_oid(&ref->old_oid)) { + !is_null_oid(&ref->old_oid) && + branch_checked_out(ref->name, &path)) { /* * If this is the head, and it's not okay to update * the head, and the old value of the head isn't empty... */ format_display(display, '!', _("[rejected]"), - wt->is_current ? - _("can't fetch in current branch") : - _("checked out in another worktree"), + path ? _("can't fetch in current branch") : + _("checked out in another worktree"), remote, pretty_ref, summary_width); + free(path); return 1; } @@ -1437,16 +1437,14 @@ static int prune_refs(struct refspec *rs, static void check_not_current_branch(struct ref *ref_map, struct worktree **worktrees) { - const struct worktree *wt; + char *path; for (; ref_map; ref_map = ref_map->next) if (ref_map->peer_ref && starts_with(ref_map->peer_ref->name, "refs/heads/") && - (wt = find_shared_symref(worktrees, "HEAD", - ref_map->peer_ref->name)) && - !wt->is_bare) + branch_checked_out(ref_map->peer_ref->name, &path)) die(_("refusing to fetch into branch '%s' " "checked out at '%s'"), - ref_map->peer_ref->name, wt->path); + ref_map->peer_ref->name, path); } static int truncate_fetch_head(void) I can extract these as patches in their own series if we think that's something we worth fast-tracking. Thanks, -Stolee ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v2 2/7] branch: add branch_checked_out() helper 2022-06-08 2:43 ` Derrick Stolee @ 2022-06-08 4:52 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-08 4:52 UTC (permalink / raw) To: Derrick Stolee Cc: Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren Derrick Stolee <derrickstolee@github.com> writes: >> Yes, that's a good plan. I'll take a look. > > Here is a fixup for this patch that should work. I'll put it in v3. > > diff --git a/branch.c b/branch.c > index 2e6419cdfa5..514212f5619 100644 > --- a/branch.c > +++ b/branch.c > @@ -10,6 +10,7 @@ > #include "worktree.h" > #include "submodule-config.h" > #include "run-command.h" > +#include "strmap.h" > > struct tracking { > struct refspec_item spec; > @@ -369,17 +370,44 @@ int validate_branchname(const char *name, struct strbuf *ref) > return ref_exists(ref->buf); > } > > -int branch_checked_out(const char *refname, char **path) > +static int initialized_checked_out_branches; > +static struct strmap current_checked_out_branches = STRMAP_INIT; > + > +static void prepare_checked_out_branches(void) > { > + int i = 0; > + struct worktree **worktrees; > + > + if (initialized_checked_out_branches) > + return; > + initialized_checked_out_branches = 1; > + > + worktrees = get_worktrees(); > + > + while (worktrees[i]) { > + struct worktree *wt = worktrees[i]; > > + if (!wt->is_bare && wt->head_ref) > + strmap_put(¤t_checked_out_branches, > + wt->head_ref, > + wt->path); > + > + i++; > + } > free_worktrees(worktrees); > - return result; > +} Yeah, the above illustrates what I had in mind pretty closely. The above outline only checks where the HEAD symref points at, and it does not protect a branch that is in the middle of getting bisected or rebased, which find_shared_symref() tries to do, IIRC, though. It should not be too hard to add that to the above to allow us to achieve feature parity with the original. > + > +int branch_checked_out(const char *refname, char **path) > +{ > + const char *path_in_set; > + prepare_checked_out_branches(); > + > + path_in_set = strmap_get(¤t_checked_out_branches, refname); > + if (path_in_set && path) > + *path = xstrdup(path_in_set); > + > + return !!path_in_set; > } This one looks quite straight-forward. > And there is another use of find_shared_symref() in the same file, allowing > us to do the following: > > diff --git a/builtin/fetch.c b/builtin/fetch.c > index ac29c2b1ae3..3933c482839 100644 > --- a/builtin/fetch.c > +++ b/builtin/fetch.c > @@ -885,7 +885,7 @@ static int update_local_ref(struct ref *ref, > struct worktree **worktrees) > { > struct commit *current = NULL, *updated; > - const struct worktree *wt; > + char *path = NULL; > const char *pretty_ref = prettify_refname(ref->name); > int fast_forward = 0; > > @@ -900,17 +900,17 @@ static int update_local_ref(struct ref *ref, > } > > if (!update_head_ok && > - (wt = find_shared_symref(worktrees, "HEAD", ref->name)) && > - !wt->is_bare && !is_null_oid(&ref->old_oid)) { > + !is_null_oid(&ref->old_oid) && > + branch_checked_out(ref->name, &path)) { Yes. It looks like a good direction to go in. Thanks. ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v2 3/7] sequencer: define array with enum values 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 1/7] log-tree: create for_each_decoration() Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 2/7] branch: add branch_checked_out() helper Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 ` Derrick Stolee via GitGitGadget 2022-06-07 22:11 ` Junio C Hamano 2022-06-07 20:42 ` [PATCH v2 4/7] sequencer: add update-refs command Derrick Stolee via GitGitGadget ` (5 subsequent siblings) 8 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The todo_command_info array defines which strings match with which todo_command enum values. The array is defined in the same order as the enum values, but if one changed without the other, then we would have unexpected results. Make it easier to see changes to the enum and this array by using the enum values as the indices of the array. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/sequencer.c b/sequencer.c index 8c3ed3532ac..8e26c9a6261 100644 --- a/sequencer.c +++ b/sequencer.c @@ -1685,20 +1685,20 @@ static struct { char c; const char *str; } todo_command_info[] = { - { 'p', "pick" }, - { 0, "revert" }, - { 'e', "edit" }, - { 'r', "reword" }, - { 'f', "fixup" }, - { 's', "squash" }, - { 'x', "exec" }, - { 'b', "break" }, - { 'l', "label" }, - { 't', "reset" }, - { 'm', "merge" }, - { 0, "noop" }, - { 'd', "drop" }, - { 0, NULL } + [TODO_PICK] = { 'p', "pick" }, + [TODO_REVERT] = { 0, "revert" }, + [TODO_EDIT] = { 'e', "edit" }, + [TODO_REWORD] = { 'r', "reword" }, + [TODO_FIXUP] = { 'f', "fixup" }, + [TODO_SQUASH] = { 's', "squash" }, + [TODO_EXEC] = { 'x', "exec" }, + [TODO_BREAK] = { 'b', "break" }, + [TODO_LABEL] = { 'l', "label" }, + [TODO_RESET] = { 't', "reset" }, + [TODO_MERGE] = { 'm', "merge" }, + [TODO_NOOP] = { 0, "noop" }, + [TODO_DROP] = { 'd', "drop" }, + [TODO_COMMENT] = { 0, NULL }, }; static const char *command_to_string(const enum todo_command command) -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v2 3/7] sequencer: define array with enum values 2022-06-07 20:42 ` [PATCH v2 3/7] sequencer: define array with enum values Derrick Stolee via GitGitGadget @ 2022-06-07 22:11 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-07 22:11 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Derrick Stolee <derrickstolee@github.com> > > The todo_command_info array defines which strings match with which > todo_command enum values. The array is defined in the same order as the > enum values, but if one changed without the other, then we would have > unexpected results. > > Make it easier to see changes to the enum and this array by using the > enum values as the indices of the array. > > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > sequencer.c | 28 ++++++++++++++-------------- > 1 file changed, 14 insertions(+), 14 deletions(-) Good thinking. I love seeing a future-proofing change like this. Thanks. > diff --git a/sequencer.c b/sequencer.c > index 8c3ed3532ac..8e26c9a6261 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -1685,20 +1685,20 @@ static struct { > char c; > const char *str; > } todo_command_info[] = { > - { 'p', "pick" }, > - { 0, "revert" }, > - { 'e', "edit" }, > - { 'r', "reword" }, > - { 'f', "fixup" }, > - { 's', "squash" }, > - { 'x', "exec" }, > - { 'b', "break" }, > - { 'l', "label" }, > - { 't', "reset" }, > - { 'm', "merge" }, > - { 0, "noop" }, > - { 'd', "drop" }, > - { 0, NULL } > + [TODO_PICK] = { 'p', "pick" }, > + [TODO_REVERT] = { 0, "revert" }, > + [TODO_EDIT] = { 'e', "edit" }, > + [TODO_REWORD] = { 'r', "reword" }, > + [TODO_FIXUP] = { 'f', "fixup" }, > + [TODO_SQUASH] = { 's', "squash" }, > + [TODO_EXEC] = { 'x', "exec" }, > + [TODO_BREAK] = { 'b', "break" }, > + [TODO_LABEL] = { 'l', "label" }, > + [TODO_RESET] = { 't', "reset" }, > + [TODO_MERGE] = { 'm', "merge" }, > + [TODO_NOOP] = { 0, "noop" }, > + [TODO_DROP] = { 'd', "drop" }, > + [TODO_COMMENT] = { 0, NULL }, > }; > > static const char *command_to_string(const enum todo_command command) ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v2 4/7] sequencer: add update-refs command 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget ` (2 preceding siblings ...) 2022-06-07 20:42 ` [PATCH v2 3/7] sequencer: define array with enum values Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 ` Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 5/7] rebase: add --update-refs option Derrick Stolee via GitGitGadget ` (4 subsequent siblings) 8 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> Add the boilerplat for an "update-refs" command in the sequencer. This connects to the current no-op do_update_refs() which will be filled in after more connections are created. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 13 ++++++++++++- sequencer.h | 1 + 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/sequencer.c b/sequencer.c index 8e26c9a6261..68f7c76e896 100644 --- a/sequencer.c +++ b/sequencer.c @@ -1696,6 +1696,7 @@ static struct { [TODO_LABEL] = { 'l', "label" }, [TODO_RESET] = { 't', "reset" }, [TODO_MERGE] = { 'm', "merge" }, + [TODO_UPDATE_REFS] = { 'u', "update-refs" }, [TODO_NOOP] = { 0, "noop" }, [TODO_DROP] = { 'd', "drop" }, [TODO_COMMENT] = { 0, NULL }, @@ -2442,7 +2443,9 @@ static int parse_insn_line(struct repository *r, struct todo_item *item, padding = strspn(bol, " \t"); bol += padding; - if (item->command == TODO_NOOP || item->command == TODO_BREAK) { + if (item->command == TODO_NOOP || + item->command == TODO_BREAK || + item->command == TODO_UPDATE_REFS) { if (bol != eol) return error(_("%s does not accept arguments: '%s'"), command_to_string(item->command), bol); @@ -4056,6 +4059,11 @@ leave_merge: return ret; } +static int do_update_refs(struct repository *r) +{ + return 0; +} + static int is_final_fixup(struct todo_list *todo_list) { int i = todo_list->current; @@ -4431,6 +4439,9 @@ static int pick_commits(struct repository *r, return error_with_patch(r, item->commit, arg, item->arg_len, opts, res, 0); + } else if (item->command == TODO_UPDATE_REFS) { + if ((res = do_update_refs(r))) + reschedule = 1; } else if (!is_noop(item->command)) return error(_("unknown command %d"), item->command); diff --git a/sequencer.h b/sequencer.h index da64473636b..c2b4e148d8f 100644 --- a/sequencer.h +++ b/sequencer.h @@ -95,6 +95,7 @@ enum todo_command { TODO_LABEL, TODO_RESET, TODO_MERGE, + TODO_UPDATE_REFS, /* commands that do nothing but are counted for reporting progress */ TODO_NOOP, TODO_DROP, -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v2 5/7] rebase: add --update-refs option 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget ` (3 preceding siblings ...) 2022-06-07 20:42 ` [PATCH v2 4/7] sequencer: add update-refs command Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 ` Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 6/7] sequencer: implement 'update-refs' command Derrick Stolee via GitGitGadget ` (3 subsequent siblings) 8 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> When working on a large feature, it can be helpful to break that feature into multiple smaller parts that become reviewed in sequence. During development or during review, a change to one part of the feature could affect multiple of these parts. An interactive rebase can help adjust the multi-part "story" of the branch. However, if there are branches tracking the different parts of the feature, then rebasing the entire list of commits can create commits not reachable from those "sub branches". It can take a manual step to update those branches. Add a new --update-refs option to 'git rebase -i' that adds 'label for-update-refs/*' steps to the todo file whenever a commit that is being rebased is decorated with that <ref>. At the very end, the 'update-refs' step is added to update all of the branches referenced by the 'label' steps. This allows the user to rebase a long list of commits in a multi-part feature and keep all of their pointers to those parts. NOTE: This change only introduce the --update-refs option and implements the changes to the todo file. It does _not_ yet implement the action taken by the 'update-refs' todo step, which will be implemented and tested in a later change. Use the new for_each_decoration() while iterating over the existing todo list. Be sure to iterate after any squashing or fixups are placed. Update the branch only after those squashes and fixups are complete. This allows a --fixup commit at the tip of the feature to apply correctly to the sub branch, even if it is fixing up the most-recent commit in that part. One potential problem here is that refs decorating commits that are already marked as "fixup!" or "squash!" will not be included in this list. Generally, the reordering of the "fixup!" and "squash!" is likely to change the relative order of these refs, so it is not recommended. The workflow here is intended to allow these kinds of commits at the tip of the rebased branch while the other sub branches come along for the ride without intervention. Be careful to not attempt updating any branch that is checked out. The most common example is the branch being rebased is checked out and decorates the tip commit. If the user is rebasing commits reachable from a different branch that is checked out in a different worktree, then they may be surprised to not see that ref update. However, it's probably best to not optimize for this scenario and do the safest thing that will result in a successful rebase. A comment is left in the TODO list that signals that these refs are currently checked out. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- Documentation/git-rebase.txt | 8 +++ builtin/rebase.c | 5 ++ sequencer.c | 105 ++++++++++++++++++++++++++++++++++ sequencer.h | 1 + t/t3404-rebase-interactive.sh | 72 +++++++++++++++++++++++ 5 files changed, 191 insertions(+) diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index 262fb01aec0..e7611b4089c 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -609,6 +609,13 @@ provided. Otherwise an explicit `--no-reschedule-failed-exec` at the start would be overridden by the presence of `rebase.rescheduleFailedExec=true` configuration. +--update-refs:: +--no-update-refs:: + Automatically force-update any branches that point to commits that + are being rebased. Any branches that are checked out in a worktree + or point to a `squash! ...` or `fixup! ...` commit are not updated + in this way. + INCOMPATIBLE OPTIONS -------------------- @@ -632,6 +639,7 @@ are incompatible with the following options: * --empty= * --reapply-cherry-picks * --edit-todo + * --update-refs * --root when used in combination with --onto In addition, the following pairs of options are incompatible: diff --git a/builtin/rebase.c b/builtin/rebase.c index 7ab50cda2ad..56d82a52106 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -102,6 +102,7 @@ struct rebase_options { int reschedule_failed_exec; int reapply_cherry_picks; int fork_point; + int update_refs; }; #define REBASE_OPTIONS_INIT { \ @@ -298,6 +299,7 @@ static int do_interactive_rebase(struct rebase_options *opts, unsigned flags) ret = complete_action(the_repository, &replay, flags, shortrevisions, opts->onto_name, opts->onto, &opts->orig_head, &commands, opts->autosquash, + opts->update_refs, &todo_list); } @@ -1124,6 +1126,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) OPT_BOOL(0, "autosquash", &options.autosquash, N_("move commits that begin with " "squash!/fixup! under -i")), + OPT_BOOL(0, "update-refs", &options.update_refs, + N_("update local refs that point to commits " + "that are being rebased")), { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"), N_("GPG-sign commits"), PARSE_OPT_OPTARG, NULL, (intptr_t) "" }, diff --git a/sequencer.c b/sequencer.c index 68f7c76e896..94f8d52e041 100644 --- a/sequencer.c +++ b/sequencer.c @@ -35,6 +35,8 @@ #include "commit-reach.h" #include "rebase-interactive.h" #include "reset.h" +#include "branch.h" +#include "log-tree.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -5614,10 +5616,110 @@ static int skip_unnecessary_picks(struct repository *r, return 0; } +struct todo_add_branch_context { + struct todo_item *items; + size_t items_nr; + size_t items_alloc; + struct strbuf *buf; + struct commit *commit; +}; + +static int add_branch_for_decoration(const struct name_decoration *d, void *data) +{ + struct todo_add_branch_context *ctx = data; + size_t base_offset = ctx->buf->len; + struct todo_item *item; + char *path; + + ALLOC_GROW(ctx->items, + ctx->items_nr + 1, + ctx->items_alloc); + item = &ctx->items[ctx->items_nr]; + memset(item, 0, sizeof(*item)); + + /* If the branch is checked out, then leave a comment instead. */ + if (branch_checked_out(d->name, &path)) { + item->command = TODO_COMMENT; + strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", + d->name, path); + free(path); + } else { + item->command = TODO_LABEL; + strbuf_addf(ctx->buf, "for-update-refs/%s\n", d->name); + } + + item->offset_in_buf = base_offset; + item->arg_offset = base_offset; + item->arg_len = ctx->buf->len - base_offset; + ctx->items_nr++; + + return 0; +} + +/* + * For each 'pick' command, find out if the commit has a decoration in + * refs/heads/. If so, then add a 'label for-update-refs/' command. + */ +static int todo_list_add_update_ref_commands(struct todo_list *todo_list) +{ + int i; + static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP; + struct decoration_filter decoration_filter = { + .include_ref_pattern = &decorate_refs_include, + .exclude_ref_pattern = &decorate_refs_exclude, + .exclude_ref_config_pattern = &decorate_refs_exclude_config + }; + struct todo_add_branch_context ctx = { + .buf = &todo_list->buf, + }; + + ctx.items_alloc = 2 * todo_list->nr + 1; + ALLOC_ARRAY(ctx.items, ctx.items_alloc); + + string_list_append(&decorate_refs_include, "refs/heads/"); + load_ref_decorations(&decoration_filter, 0); + + for (i = 0; i < todo_list->nr; ) { + struct todo_item *item = &todo_list->items[i]; + + /* insert ith item into new list */ + ALLOC_GROW(ctx.items, + ctx.items_nr + 1, + ctx.items_alloc); + + ctx.items[ctx.items_nr++] = todo_list->items[i++]; + + if (item->commit) { + ctx.commit = item->commit; + for_each_decoration(item->commit, + add_branch_for_decoration, + &ctx); + } + } + + /* Add the "update-refs" step. */ + ALLOC_GROW(ctx.items, + ctx.items_nr + 1, + ctx.items_alloc); + memset(&ctx.items[ctx.items_nr], 0, sizeof(struct todo_item)); + ctx.items[ctx.items_nr].command = TODO_UPDATE_REFS; + ctx.items_nr++; + + free(todo_list->items); + todo_list->items = ctx.items; + todo_list->nr = ctx.items_nr; + todo_list->alloc = ctx.items_alloc; + + return 0; +} + int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags, const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, + unsigned update_refs, struct todo_list *todo_list) { char shortonto[GIT_MAX_HEXSZ + 1]; @@ -5636,6 +5738,9 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla item->arg_len = item->arg_offset = item->flags = item->offset_in_buf = 0; } + if (update_refs && todo_list_add_update_ref_commands(todo_list)) + return -1; + if (autosquash && todo_list_rearrange_squash(todo_list)) return -1; diff --git a/sequencer.h b/sequencer.h index c2b4e148d8f..e268208b315 100644 --- a/sequencer.h +++ b/sequencer.h @@ -167,6 +167,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, + unsigned update_refs, struct todo_list *todo_list); int todo_list_rearrange_squash(struct todo_list *todo_list); diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index f31afd4a547..5e99ad7f3b6 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1743,6 +1743,78 @@ test_expect_success 'ORIG_HEAD is updated correctly' ' test_cmp_rev ORIG_HEAD test-orig-head@{1} ' +test_expect_success '--update-refs adds label and update-ref commands' ' + git checkout -b update-refs no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git commit --allow-empty --fixup=third && + git branch -f shared-tip && + ( + set_cat_todo_editor && + + cat >expect <<-EOF && + pick $(git log -1 --format=%h J) J + label for-update-refs/refs/heads/second + label for-update-refs/refs/heads/first + pick $(git log -1 --format=%h K) K + pick $(git log -1 --format=%h L) L + fixup $(git log -1 --format=%h update-refs) fixup! L # empty + label for-update-refs/refs/heads/third + pick $(git log -1 --format=%h M) M + label for-update-refs/refs/heads/no-conflict-branch + label for-update-refs/refs/heads/shared-tip + update-refs + EOF + + test_must_fail git rebase -i --autosquash --update-refs primary >todo && + test_cmp expect todo + ) +' + +test_expect_success '--update-refs adds commands with --rebase-merges' ' + git checkout -b update-refs-with-merge no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git merge -m merge branch2 && + git branch -f merge-branch && + git commit --fixup=third --allow-empty && + ( + set_cat_todo_editor && + + cat >expect <<-EOF && + label onto + reset onto + pick $(git log -1 --format=%h branch2~1) F + pick $(git log -1 --format=%h branch2) I + label for-update-refs/refs/heads/branch2 + label merge + reset onto + pick $(git log -1 --format=%h refs/heads/second) J + label for-update-refs/refs/heads/second + label for-update-refs/refs/heads/first + pick $(git log -1 --format=%h refs/heads/third~1) K + pick $(git log -1 --format=%h refs/heads/third) L + fixup $(git log -1 --format=%h update-refs-with-merge) fixup! L # empty + label for-update-refs/refs/heads/third + pick $(git log -1 --format=%h HEAD~2) M + label for-update-refs/refs/heads/no-conflict-branch + merge -C $(git log -1 --format=%h HEAD~1) merge # merge + label for-update-refs/refs/heads/merge-branch + update-refs + EOF + + test_must_fail git rebase -i --autosquash \ + --rebase-merges=rebase-cousins \ + --update-refs primary >todo && + + test_cmp expect todo + ) +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v2 6/7] sequencer: implement 'update-refs' command 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget ` (4 preceding siblings ...) 2022-06-07 20:42 ` [PATCH v2 5/7] rebase: add --update-refs option Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 ` Derrick Stolee via GitGitGadget 2022-06-07 22:23 ` Junio C Hamano 2022-06-07 20:42 ` [PATCH v2 7/7] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget ` (2 subsequent siblings) 8 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The previous change allowed 'git rebase --update-refs' to create 'label' commands for each branch among the commits being rewritten and add an 'update-refs' command at the end of the todo list. Now, teach Git to update the refs during that final 'update-refs' command. We need to create an array of new and old OIDs for each ref by iterating over the refs/rewritten/for-update-refs/ namespace. We cannot update the refs in-place since this will confuse the refs iterator. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 62 ++++++++++++++++++++++++++++++++++- t/t3404-rebase-interactive.sh | 24 ++++++++++++++ 2 files changed, 85 insertions(+), 1 deletion(-) diff --git a/sequencer.c b/sequencer.c index 94f8d52e041..a8f62ce8e5f 100644 --- a/sequencer.c +++ b/sequencer.c @@ -4061,11 +4061,71 @@ leave_merge: return ret; } -static int do_update_refs(struct repository *r) +struct update_refs_context { + struct ref_store *refs; + char **ref_names; + struct object_id *old; + struct object_id *new; + size_t nr; + size_t alloc; +}; + +static int add_ref_to_context(const char *refname, + const struct object_id *oid, + int flags, + void *data) { + int f = 0; + const char *name; + struct update_refs_context *ctx = data; + + ALLOC_GROW(ctx->ref_names, ctx->nr + 1, ctx->alloc); + ALLOC_GROW(ctx->old, ctx->nr + 1, ctx->alloc); + ALLOC_GROW(ctx->new, ctx->nr + 1, ctx->alloc); + + if (!skip_prefix(refname, "refs/rewritten/for-update-refs/", &name)) + return 1; + + ctx->ref_names[ctx->nr] = xstrdup(name); + oidcpy(&ctx->new[ctx->nr], oid); + if (!refs_resolve_ref_unsafe(ctx->refs, name, 0, + &ctx->old[ctx->nr], &f)) + return 1; + + ctx->nr++; return 0; } +static int do_update_refs(struct repository *r) +{ + int i, res; + struct update_refs_context ctx = { + .refs = get_main_ref_store(r), + .alloc = 16, + }; + ALLOC_ARRAY(ctx.ref_names, ctx.alloc); + ALLOC_ARRAY(ctx.old, ctx.alloc); + ALLOC_ARRAY(ctx.new, ctx.alloc); + + res = refs_for_each_fullref_in(ctx.refs, + "refs/rewritten/for-update-refs/", + add_ref_to_context, + &ctx); + + for (i = 0; !res && i < ctx.nr; i++) + res = refs_update_ref(ctx.refs, "rewritten during rebase", + ctx.ref_names[i], + &ctx.new[i], &ctx.old[i], + 0, UPDATE_REFS_MSG_ON_ERR); + + for (i = 0; i < ctx.nr; i++) + free(ctx.ref_names[i]); + free(ctx.ref_names); + free(ctx.old); + free(ctx.new); + return res; +} + static int is_final_fixup(struct todo_list *todo_list) { int i = todo_list->current; diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 5e99ad7f3b6..72711efec28 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1815,6 +1815,30 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' ) ' +compare_two_refs () { + git rev-parse $1 >expect && + git rev-parse $2 >actual && + test_cmp expect actual +} + +test_expect_success '--update-refs updates refs correctly' ' + git checkout -B update-refs no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + test_commit extra2 fileX && + git commit --amend --fixup=L && + ( + git rebase -i --autosquash --update-refs primary && + + compare_two_refs HEAD~3 refs/heads/first && + compare_two_refs HEAD~3 refs/heads/second && + compare_two_refs HEAD~1 refs/heads/third && + compare_two_refs HEAD refs/heads/no-conflict-branch + ) +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v2 6/7] sequencer: implement 'update-refs' command 2022-06-07 20:42 ` [PATCH v2 6/7] sequencer: implement 'update-refs' command Derrick Stolee via GitGitGadget @ 2022-06-07 22:23 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-07 22:23 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Derrick Stolee <derrickstolee@github.com> > > The previous change allowed 'git rebase --update-refs' to create 'label' > commands for each branch among the commits being rewritten and add an > 'update-refs' command at the end of the todo list. Now, teach Git to > update the refs during that final 'update-refs' command. > > We need to create an array of new and old OIDs for each ref by iterating > over the refs/rewritten/for-update-refs/ namespace. We cannot update the > refs in-place since this will confuse the refs iterator. In other words, grab everything we need to do and then use that in-core information to tell refs API what refs to change to what values? That dounds like a quite reasonable thing to do. Looking at the patch text, the only thing that stands out at me is that this does not seem to perform the updates in a single transaction, which may often not matter in practice, but may be a prudent thing to do anyway at philosophical level. Thanks, will queue. ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v2 7/7] rebase: add rebase.updateRefs config option 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget ` (5 preceding siblings ...) 2022-06-07 20:42 ` [PATCH v2 6/7] sequencer: implement 'update-refs' command Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 ` Derrick Stolee via GitGitGadget 2022-06-08 14:32 ` [PATCH v2 0/7] rebase: update branches in multi-part topic Phillip Wood 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget 8 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-07 20:42 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The previous change added the --update-refs command-line option. For users who always want this mode, create the rebase.updateRefs config option which behaves the same way as rebase.autoSquash does with the --autosquash option. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- Documentation/config/rebase.txt | 3 +++ Documentation/git-rebase.txt | 4 ++++ builtin/rebase.c | 5 +++++ t/t3404-rebase-interactive.sh | 14 ++++++++++++++ 4 files changed, 26 insertions(+) diff --git a/Documentation/config/rebase.txt b/Documentation/config/rebase.txt index 8c979cb20f2..f19bd0e0407 100644 --- a/Documentation/config/rebase.txt +++ b/Documentation/config/rebase.txt @@ -21,6 +21,9 @@ rebase.autoStash:: `--autostash` options of linkgit:git-rebase[1]. Defaults to false. +rebase.updateRefs:: + If set to true enable `--update-refs` option by default. + rebase.missingCommitsCheck:: If set to "warn", git rebase -i will print a warning if some commits are removed (e.g. a line was deleted), however the diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index e7611b4089c..7a56dbb9bec 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -615,6 +615,10 @@ start would be overridden by the presence of are being rebased. Any branches that are checked out in a worktree or point to a `squash! ...` or `fixup! ...` commit are not updated in this way. ++ +If the `--update-refs` option is enabled by default using the +configuration variable `rebase.updateRefs`, this option can be +used to override and disable this setting. INCOMPATIBLE OPTIONS -------------------- diff --git a/builtin/rebase.c b/builtin/rebase.c index 56d82a52106..8ebc98ea505 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -802,6 +802,11 @@ static int rebase_config(const char *var, const char *value, void *data) return 0; } + if (!strcmp(var, "rebase.updaterefs")) { + opts->update_refs = git_config_bool(var, value); + return 0; + } + if (!strcmp(var, "rebase.reschedulefailedexec")) { opts->reschedule_failed_exec = git_config_bool(var, value); return 0; diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 72711efec28..f580afd8723 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1769,6 +1769,12 @@ test_expect_success '--update-refs adds label and update-ref commands' ' EOF test_must_fail git rebase -i --autosquash --update-refs primary >todo && + test_cmp expect todo && + + test_must_fail git -c rebase.autosquash=true \ + -c rebase.updaterefs=true \ + rebase -i primary >todo && + test_cmp expect todo ) ' @@ -1811,6 +1817,14 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' --rebase-merges=rebase-cousins \ --update-refs primary >todo && + test_cmp expect todo && + + test_must_fail git -c rebase.autosquash=true \ + -c rebase.updaterefs=true \ + rebase -i \ + --rebase-merges=rebase-cousins \ + primary >todo && + test_cmp expect todo ) ' -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v2 0/7] rebase: update branches in multi-part topic 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget ` (6 preceding siblings ...) 2022-06-07 20:42 ` [PATCH v2 7/7] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget @ 2022-06-08 14:32 ` Phillip Wood 2022-06-08 18:09 ` Derrick Stolee 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget 8 siblings, 1 reply; 144+ messages in thread From: Phillip Wood @ 2022-06-08 14:32 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee Hi Stolee On 07/06/2022 21:42, Derrick Stolee via GitGitGadget wrote: > [...] > As an example, here is my in-progress bundle URI RFC split into subtopics as > they appear during the TODO list of a git rebase -i --update-refs: > > pick 2d966282ff3 docs: document bundle URI standard > pick 31396e9171a remote-curl: add 'get' capability > pick 54c6ab70f67 bundle-uri: create basic file-copy logic > pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// > pick 6adaf842684 fetch: add --bundle-uri option > pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration > label for-update-refs/refs/heads/bundle-redo/fetch > [...] > update-refs > [...] > Based on some of the discussion, it seemed like one way to do this would be > to have an 'update-ref ' command that would take the place of these 'label' > commands. However, this would require two things that make it a bit awkward: > > 1. We would need to replicate the storage of those positions during the > rebase. 'label' already does this pretty well. I've added the > "for-update-refs/" label to help here. I'm afraid I don't think that using a label with a name constructed from a magic prefix and the full refname is a good user interface. (i) Having labels behave differently based on their name is confusing. (ii) The length of the label string is cumbersome for users. Rebase already has a reputation for being unfriendly and hard to use, this will not improve that. "update-ref bundle-redo/fetch" is much simpler. (iii) It means that we no longer store the oid of each branch when creating the todo list and so cannot check if it is safe to update it at the end of the rebase (just using the current value of the branch ref at the end of a long running operation like rebase is not sufficient to be safe). > 2. If we want to close out all of the refs as the rebase is finishing, then > that "step" becomes invisible to the user (and a bit more complicated to > insert). Thus, the 'update-refs' step performs this action. If the user > wants to do things after that step, then they can do so by editing the > TODO list. I'm not sure what the use case is for making the update-refs step visible to the user - it seems to be added complexity for them to deal with. We don't do that for the branch that is being rebased so why do we need to do it for the other branches? As for the implementation we could just update the branches at the end of the loop in pick_commits() where we update the branch and run the post-rewrite hook etc. there is no need for an entry in the todo list. Best Wishes Phillip ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v2 0/7] rebase: update branches in multi-part topic 2022-06-08 14:32 ` [PATCH v2 0/7] rebase: update branches in multi-part topic Phillip Wood @ 2022-06-08 18:09 ` Derrick Stolee 2022-06-09 10:04 ` Phillip Wood 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-06-08 18:09 UTC (permalink / raw) To: phillip.wood, Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren On 6/8/2022 10:32 AM, Phillip Wood wrote: > Hi Stolee > > On 07/06/2022 21:42, Derrick Stolee via GitGitGadget wrote: >> [...] >> As an example, here is my in-progress bundle URI RFC split into subtopics as >> they appear during the TODO list of a git rebase -i --update-refs: >> >> pick 2d966282ff3 docs: document bundle URI standard >> pick 31396e9171a remote-curl: add 'get' capability >> pick 54c6ab70f67 bundle-uri: create basic file-copy logic >> pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// >> pick 6adaf842684 fetch: add --bundle-uri option >> pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration >> label for-update-refs/refs/heads/bundle-redo/fetch >> [...] update-refs >> [...] >> Based on some of the discussion, it seemed like one way to do this would be >> to have an 'update-ref ' command that would take the place of these 'label' >> commands. However, this would require two things that make it a bit awkward: >> >> 1. We would need to replicate the storage of those positions during the >> rebase. 'label' already does this pretty well. I've added the >> "for-update-refs/" label to help here. > > I'm afraid I don't think that using a label with a name constructed from > a magic prefix and the full refname is a good user interface. > > (i) Having labels behave differently based on their name is confusing. The label commands do as advertised, but the 'update-refs' command does something with the set of refs based on their names, yes. We would need to store this information during the rebase _somewhere_, and refs/rewritten/ seems natural. > (ii) The length of the label string is cumbersome for users. Rebase already > has a reputation for being unfriendly and hard to use, this will not improve > that. "update-ref bundle-redo/fetch" is much simpler. I agree that your proposed replacement for "label for-update-refs/..." would look cleaner. I don't think that that is enough to justify hiding information from the user. > (iii) It means that we no longer store the oid of each branch when creating> the todo list and so cannot check if it is safe to update it at the end of the > rebase (just using the current value of the branch ref at the end of a long > running operation like rebase is not sufficient to be safe). The operation we are doing necessarily requires a force-update, since we are modifying history. The safety you are caring about here is about the case where the user modifies one of these refs between the initial 'git rebase --update-refs' and the final 'git rebase --continue' that actually writes the refs. You want to prevent that final update from succeeding because another change to those refs could be lost. I'm less concerned about this case, because the user is requesting that we update the refs pointing to this set of commits. But, I'm glad you brought it up. One way to prevent this situation would be to extend the idea of "what branch is being rebased?" (REBASE_HEAD) to "which branches are being rewritten?" (REBASE_HEADS?). If the worktree kept the full list somewhere in the $GIT_DIR, then other Git commands could notice that those refs are currently being overwritten and those updates should fail (similarly to how we currently fail to update a branch being rebased). This ties into your later point: >> 2. If we want to close out all of the refs as the rebase is finishing, then >> that "step" becomes invisible to the user (and a bit more complicated to >> insert). Thus, the 'update-refs' step performs this action. If the user >> wants to do things after that step, then they can do so by editing the >> TODO list. > > I'm not sure what the use case is for making the update-refs step visible to > the user - it seems to be added complexity for them to deal with. We don't do > that for the branch that is being rebased so why do we need to do it for the > other branches? As for the implementation we could just update the branches > at the end of the loop in pick_commits() where we update the branch and run the > post-rewrite hook etc. there is no need for an entry in the todo list. I concede that allowing the user to move the 'update-refs' command around is not super important. The biggest concern I originally had with this idea was that there was no connection between the "--update-refs" option in the first "git rebase" command and the final "git rebase --continue" command. That is, other than which refs are in refs/rewritten/*. HOWEVER, using refs/rewritten/* is likely buggy: if we had two rebases happening at the same time (operating on a disjoint set of refs), then how do we differentiate which refs make sense for us to update as we complete this rebase? So, I'm coming to the conclusion that using refs/rewritten/* is problematic and I should use something closer to REBASE_HEAD as a way to describe which refs are being updated. In that context, your 'update-ref <ref>' command makes a lot more sense. The user can decide to move those around _or_ remove them; it won't change their protection under "REBASE_HEADS" but would prevent them from being rewritten. While I think on this, I'll send my branch_checked_out() patches in a separate thread, since those have grown in complexity since this version. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v2 0/7] rebase: update branches in multi-part topic 2022-06-08 18:09 ` Derrick Stolee @ 2022-06-09 10:04 ` Phillip Wood 0 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-06-09 10:04 UTC (permalink / raw) To: Derrick Stolee, phillip.wood, Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren Hi Stolee On 08/06/2022 19:09, Derrick Stolee wrote: > On 6/8/2022 10:32 AM, Phillip Wood wrote: >> Hi Stolee >> >> On 07/06/2022 21:42, Derrick Stolee via GitGitGadget wrote: >>> [...] >>> As an example, here is my in-progress bundle URI RFC split into subtopics as >>> they appear during the TODO list of a git rebase -i --update-refs: >>> >>> pick 2d966282ff3 docs: document bundle URI standard >>> pick 31396e9171a remote-curl: add 'get' capability >>> pick 54c6ab70f67 bundle-uri: create basic file-copy logic >>> pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// >>> pick 6adaf842684 fetch: add --bundle-uri option >>> pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration >>> label for-update-refs/refs/heads/bundle-redo/fetch >>> [...] update-refs >>> [...] >>> Based on some of the discussion, it seemed like one way to do this would be >>> to have an 'update-ref ' command that would take the place of these 'label' >>> commands. However, this would require two things that make it a bit awkward: >>> >>> 1. We would need to replicate the storage of those positions during the >>> rebase. 'label' already does this pretty well. I've added the >>> "for-update-refs/" label to help here. >> >> I'm afraid I don't think that using a label with a name constructed from >> a magic prefix and the full refname is a good user interface. >> >> (i) Having labels behave differently based on their name is confusing. > > The label commands do as advertised, but the 'update-refs' command does > something with the set of refs based on their names, yes. We would need to > store this information during the rebase _somewhere_, and refs/rewritten/ > seems natural. > >> (ii) The length of the label string is cumbersome for users. Rebase already >> has a reputation for being unfriendly and hard to use, this will not improve >> that. "update-ref bundle-redo/fetch" is much simpler. > > I agree that your proposed replacement for "label for-update-refs/..." would > look cleaner. I don't think that that is enough to justify hiding information > from the user. In general I think it is better not to burden the user with unnecessary information - it makes the ui more complicated and exposes implementation details which makes it harder to change the implementation. >> (iii) It means that we no longer store the oid of each branch when creating> the todo list and so cannot check if it is safe to update it at the end of the >> rebase (just using the current value of the branch ref at the end of a long >> running operation like rebase is not sufficient to be safe). > > The operation we are doing necessarily requires a force-update, since we are > modifying history. The safety you are caring about here is about the case where > the user modifies one of these refs between the initial 'git rebase --update-refs' > and the final 'git rebase --continue' that actually writes the refs. You want to > prevent that final update from succeeding because another change to those refs > could be lost. > > I'm less concerned about this case, because the user is requesting that we > update the refs pointing to this set of commits. But, I'm glad you brought it > up. At the end of the rebase we pass the oid of the branch that we store at the start of the rebase when updating the ref to avoid nasty surprises. I think it makes sense to do the same for the other branches being rewritten - it is easy to get stuck on a conflict resolution and go and do something else for a while and forget a branch is being rebased. If we cannot update the ref at the end of the rebase we should print the new oid with instructions to run "git reset --hard" or "git update-ref" if the user wants to update the branch. > One way to prevent this situation would be to extend the idea of "what branch > is being rebased?" (REBASE_HEAD) to "which branches are being rewritten?" > (REBASE_HEADS?). Just to clarify REBASE_HEAD points to the commit we're currently trying to pick, the branch ref is stored it .git/rebase-merge/head-name and its oid in .git/rebase-merge/orig-head > If the worktree kept the full list somewhere in the $GIT_DIR, > then other Git commands could notice that those refs are currently being > overwritten and those updates should fail (similarly to how we currently fail > to update a branch being rebased). That's a good point and one that I should have remembered as I implemented that check in my script. I agree that we should stop the user starting a rebase in a worktree whose branch is being updated elsewhere. If we write a list of the refs we want to update under .git/rebase-merge before walking the other worktrees to see what's being rebased elsewhere that avoids a race where two processes start that try to rebase the same branch in different worktrees at the same time. It would be easy to store the original oids with the ref names. An added complexity is that we would have to check the todo list for any new update-ref commands and check those refs are not being rebased elsewhere each time the list is edited. In the future we should update "git status" to read that file but I don't think that needs to be part of the initial implementation. > This ties into your later point: > >>> 2. If we want to close out all of the refs as the rebase is finishing, then >>> that "step" becomes invisible to the user (and a bit more complicated to >>> insert). Thus, the 'update-refs' step performs this action. If the user >>> wants to do things after that step, then they can do so by editing the >>> TODO list. >> >> I'm not sure what the use case is for making the update-refs step visible to >> the user - it seems to be added complexity for them to deal with. We don't do >> that for the branch that is being rebased so why do we need to do it for the >> other branches? As for the implementation we could just update the branches >> at the end of the loop in pick_commits() where we update the branch and run the >> post-rewrite hook etc. there is no need for an entry in the todo list. > > I concede that allowing the user to move the 'update-refs' command around is > not super important. > > The biggest concern I originally had with this idea was that there was no > connection between the "--update-refs" option in the first "git rebase" > command and the final "git rebase --continue" command. That is, other than > which refs are in refs/rewritten/*. > > HOWEVER, using refs/rewritten/* is likely buggy: if we had two rebases > happening at the same time (operating on a disjoint set of refs), then how > do we differentiate which refs make sense for us to update as we complete > this rebase? >> So, I'm coming to the conclusion that using refs/rewritten/* is problematic > and I should use something closer to REBASE_HEAD as a way to describe which > refs are being updated. In that context, your 'update-ref <ref>' command > makes a lot more sense. The user can decide to move those around _or_ remove > them; it won't change their protection under "REBASE_HEADS" but would > prevent them from being rewritten. Whenever one adds a new option to rebase it almost always involves writing more state to .git/rebase-merge, in this case I think we want to store the branches from the update-ref commands and their original oids there. I'm not sure that we want to expose the file to the user though so we can change the implementation in the future if we need to (e.g. we may decide it is better to store this information for all worktrees under a single file in $GIT_COMMON_DIR). > While I think on this, I'll send my branch_checked_out() patches in a > separate thread, since those have grown in complexity since this version. If you want to discuss any ideas face-to-face drop me a email and we can arrange a time to chat if that would be useful. Best Wishes Phillip > -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v3 0/8] rebase: update branches in multi-part topic 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget ` (7 preceding siblings ...) 2022-06-08 14:32 ` [PATCH v2 0/7] rebase: update branches in multi-part topic Phillip Wood @ 2022-06-28 13:25 ` Derrick Stolee via GitGitGadget 2022-06-28 13:25 ` [PATCH v3 1/8] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget ` (10 more replies) 8 siblings, 11 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee (Update: This is now based on ds/branch-checked-out.) This is a feature I've wanted for quite a while. When working on the sparse index topic, I created a long RFC that actually broke into three topics for full review upstream. These topics were sequential, so any feedback on an earlier one required updates to the later ones. I would work on the full feature and use interactive rebase to update the full list of commits. However, I would need to update the branches pointing to those sub-topics. This series adds a new --update-refs option to 'git rebase' (along with a rebase.updateRefs config option) that adds 'update-ref' commands into the TODO list. This is powered by the commit decoration machinery. As an example, here is my in-progress bundle URI RFC split into subtopics as they appear during the TODO list of a git rebase -i --update-refs: pick 2d966282ff3 docs: document bundle URI standard pick 31396e9171a remote-curl: add 'get' capability pick 54c6ab70f67 bundle-uri: create basic file-copy logic pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// pick 6adaf842684 fetch: add --bundle-uri option pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration update-ref refs/heads/bundle-redo/fetch pick 1e3f6546632 clone: add --bundle-uri option pick 9e4a6fe9b68 clone: --bundle-uri cannot be combined with --depth update-ref refs/heads/bundle-redo/clone pick 5451cb6599c bundle-uri: create bundle_list struct and helpers pick 3029c3aca15 bundle-uri: create base key-value pair parsing pick a8b2de79ce8 bundle-uri: create "key=value" line parsing pick 92625a47673 bundle-uri: unit test "key=value" parsing pick a8616af4dc2 bundle-uri: limit recursion depth for bundle lists pick 9d6809a8d53 bundle-uri: parse bundle list in config format pick 287a732b54c bundle-uri: fetch a list of bundles update-ref refs/heads/bundle-redo/list pick b09f8226185 protocol v2: add server-side "bundle-uri" skeleton pick 520204dcd1c bundle-uri client: add minimal NOOP client pick 62e8b457b48 bundle-uri client: add "git ls-remote-bundle-uri" pick 00eae925043 bundle-uri: serve URI advertisement from bundle.* config pick 4277440a250 bundle-uri client: add boolean transfer.bundleURI setting pick caf4599a81d bundle-uri: allow relative URLs in bundle lists pick df255000b7e bundle-uri: download bundles from an advertised list pick d71beabf199 clone: unbundle the advertised bundles pick c9578391976 t5601: basic bundle URI tests # Ref refs/heads/bundle-redo/rfc-3 checked out at '/home/stolee/_git/git-bundles' update-ref refs/heads/bundle-redo/advertise Here is an outline of the series: * Patch 1 updates some tests for branch_checked_out() for the 'apply' backend. * Patch 2 updates branch_checked_out() to parse the rebase-merge/update-refs file to block concurrent ref updates and checkouts on branches selected by --update-refs. * Patch 3 updates the todo list documentation to remove some unnecessary dots in the 'merge' command. This makes it consistent with the 'fixup' command before we document the 'update-ref' command. * Patch 4 updates the definition of todo_command_info to use enum values as array indices. * Patches 5-7 implement the --update-refs logic itself. * Patch 8 adds the rebase.updateRefs config option similar to rebase.autoSquash. Updates in v3 ============= * The branch_checked_out() API was extracted to its own topic and is now the ds/branch-checked-out branch. This series is now based on that one. * The for_each_decoration() API was removed, since it became trivial once it did not take a commit directly. * The branch_checked_out() tests did not verify the rebase-apply data (for the apply backend), so that is fixed. * Instead of using the 'label' command and a final 'update-refs' command in the todo list, use a new 'update-ref ' command. This command updates the rebase-merge/update-refs file with the OID of HEAD at these steps. At the very end of the rebase sequence, those refs are updated to the stored OID values (assuming that they were not removed by the user, in which case we notice that the OID is the null OID and we do nothing). * New tests are added. * The todo-list comment documentation has some new formatting updates, but also includes a description of 'update-refs' in this version. Updates in v2 ============= As recommended by the excellent feedback, I have removed the 'exec' commands in favor of the 'label' commands and a new 'update-refs' command at the very end. This way, there is only one step that updates all of the refs at the end instead of updating refs during the rebase. If a user runs 'git rebase --abort' in the middle, then their refs are still where they need to be. Based on some of the discussion, it seemed like one way to do this would be to have an 'update-ref ' command that would take the place of these 'label' commands. However, this would require two things that make it a bit awkward: 1. We would need to replicate the storage of those positions during the rebase. 'label' already does this pretty well. I've added the "for-update-refs/" label to help here. 2. If we want to close out all of the refs as the rebase is finishing, then that "step" becomes invisible to the user (and a bit more complicated to insert). Thus, the 'update-refs' step performs this action. If the user wants to do things after that step, then they can do so by editing the TODO list. Other updates: * The 'keep_decorations' parameter was renamed to 'update_refs'. * I added tests for --rebase-merges=rebase-cousins to show how these labels interact with other labels and merge commands. * I changed the order of the insertion of these update-refs labels to be before the fixups are rearranged. This fixes a bug where the tip commit is a fixup! so its decorations are never inspected (and they would be in the wrong place even if they were). The fixup! commands are properly inserted between a pick and its following label command. Tests demonstrate this is correct. * Numerous style choices are updated based on feedback. Thank you for all of the detailed review and ideas in this space. I appreciate any more ideas that can make this feature as effective as it can be. Thanks, -Stolee Derrick Stolee (8): t2407: test branches currently using apply backend branch: consider refs under 'update-refs' rebase-interactive: update 'merge' description sequencer: define array with enum values sequencer: add update-ref command rebase: add --update-refs option rebase: update refs from 'update-ref' commands rebase: add rebase.updateRefs config option Documentation/config/rebase.txt | 3 + Documentation/git-rebase.txt | 11 ++ branch.c | 13 ++ builtin/rebase.c | 10 ++ rebase-interactive.c | 9 +- sequencer.c | 303 ++++++++++++++++++++++++++++++-- sequencer.h | 11 ++ t/t2407-worktree-heads.sh | 52 +++++- t/t3404-rebase-interactive.sh | 107 +++++++++++ 9 files changed, 496 insertions(+), 23 deletions(-) base-commit: 9bef0b1e6ec371e786c2fba3edcc06ad040a536c Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1247%2Fderrickstolee%2Frebase-keep-decorations-v3 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1247/derrickstolee/rebase-keep-decorations-v3 Pull-Request: https://github.com/gitgitgadget/git/pull/1247 Range-diff vs v2: 1: 4f9f3487641 < -: ----------- log-tree: create for_each_decoration() 2: 5f54766e103 < -: ----------- branch: add branch_checked_out() helper -: ----------- > 1: fbaedc7f1f0 t2407: test branches currently using apply backend -: ----------- > 2: 2bc647b6fcd branch: consider refs under 'update-refs' -: ----------- > 3: 669f4abd59e rebase-interactive: update 'merge' description 3: 9f261c7df2c = 4: 6528a50343f sequencer: define array with enum values 4: 842b2186d25 ! 5: e95ad41d355 sequencer: add update-refs command @@ Metadata Author: Derrick Stolee <derrickstolee@github.com> ## Commit message ## - sequencer: add update-refs command + sequencer: add update-ref command - Add the boilerplat for an "update-refs" command in the sequencer. This - connects to the current no-op do_update_refs() which will be filled in + Add the boilerplate for an "update-ref" command in the sequencer. This + connects to the current no-op do_update_ref() which will be filled in after more connections are created. + The syntax in the todo list will be "update-ref <ref-name>" to signal + that we should store the current commit as the value for updating + <ref-name> at the end of the rebase. + Signed-off-by: Derrick Stolee <derrickstolee@github.com> + ## rebase-interactive.c ## +@@ rebase-interactive.c: void append_todo_help(int command_count, + " create a merge commit using the original merge commit's\n" + " message (or the oneline, if no original merge commit was\n" + " specified); use -c <commit> to reword the commit message\n" ++"u, update-ref <ref> = track a placeholder for the <ref> to be updated\n" ++" to this position in the new commits. The <ref> is\n" ++" updated at the end of the rebase\n" + "\n" + "These lines can be re-ordered; they are executed from top to bottom.\n"); + unsigned edit_todo = !(shortrevisions && shortonto); + ## sequencer.c ## @@ sequencer.c: static struct { [TODO_LABEL] = { 'l', "label" }, [TODO_RESET] = { 't', "reset" }, [TODO_MERGE] = { 'm', "merge" }, -+ [TODO_UPDATE_REFS] = { 'u', "update-refs" }, ++ [TODO_UPDATE_REF] = { 'u', "update-ref" }, [TODO_NOOP] = { 0, "noop" }, [TODO_DROP] = { 'd', "drop" }, [TODO_COMMENT] = { 0, NULL }, @@ sequencer.c: static int parse_insn_line(struct repository *r, struct todo_item *item, - padding = strspn(bol, " \t"); - bol += padding; + command_to_string(item->command)); -- if (item->command == TODO_NOOP || item->command == TODO_BREAK) { -+ if (item->command == TODO_NOOP || -+ item->command == TODO_BREAK || -+ item->command == TODO_UPDATE_REFS) { - if (bol != eol) - return error(_("%s does not accept arguments: '%s'"), - command_to_string(item->command), bol); + if (item->command == TODO_EXEC || item->command == TODO_LABEL || +- item->command == TODO_RESET) { ++ item->command == TODO_RESET || item->command == TODO_UPDATE_REF) { + item->commit = NULL; + item->arg_offset = bol - buf; + item->arg_len = (int)(eol - bol); @@ sequencer.c: leave_merge: return ret; } -+static int do_update_refs(struct repository *r) ++static int do_update_ref(struct repository *r, const char *ref_name) +{ + return 0; +} @@ sequencer.c: static int pick_commits(struct repository *r, return error_with_patch(r, item->commit, arg, item->arg_len, opts, res, 0); -+ } else if (item->command == TODO_UPDATE_REFS) { -+ if ((res = do_update_refs(r))) ++ } else if (item->command == TODO_UPDATE_REF) { ++ struct strbuf ref = STRBUF_INIT; ++ strbuf_add(&ref, arg, item->arg_len); ++ if ((res = do_update_ref(r, ref.buf))) + reschedule = 1; ++ strbuf_release(&ref); } else if (!is_noop(item->command)) return error(_("unknown command %d"), item->command); @@ sequencer.h: enum todo_command { TODO_LABEL, TODO_RESET, TODO_MERGE, -+ TODO_UPDATE_REFS, ++ TODO_UPDATE_REF, /* commands that do nothing but are counted for reporting progress */ TODO_NOOP, TODO_DROP, 5: 0a4c110127b ! 6: 918b398d6a2 rebase: add --update-refs option @@ Commit message reachable from those "sub branches". It can take a manual step to update those branches. - Add a new --update-refs option to 'git rebase -i' that adds 'label - for-update-refs/*' steps to the todo file whenever a commit that is - being rebased is decorated with that <ref>. At the very end, the - 'update-refs' step is added to update all of the branches referenced by - the 'label' steps. This allows the user to rebase a long list of commits - in a multi-part feature and keep all of their pointers to those parts. + Add a new --update-refs option to 'git rebase -i' that adds 'update-ref + <ref>' steps to the todo file whenever a commit that is being rebased is + decorated with that <ref>. At the very end, the rebase process updates + all of the listed refs to the values stored during the rebase operation. - NOTE: This change only introduce the --update-refs option and implements - the changes to the todo file. It does _not_ yet implement the action - taken by the 'update-refs' todo step, which will be implemented and - tested in a later change. - - Use the new for_each_decoration() while iterating over the existing todo - list. Be sure to iterate after any squashing or fixups are placed. - Update the branch only after those squashes and fixups are complete. - This allows a --fixup commit at the tip of the feature to apply - correctly to the sub branch, even if it is fixing up the most-recent - commit in that part. + Be sure to iterate after any squashing or fixups are placed. Update the + branch only after those squashes and fixups are complete. This allows a + --fixup commit at the tip of the feature to apply correctly to the sub + branch, even if it is fixing up the most-recent commit in that part. One potential problem here is that refs decorating commits that are already marked as "fixup!" or "squash!" will not be included in this @@ Commit message of the rebased branch while the other sub branches come along for the ride without intervention. - Be careful to not attempt updating any branch that is checked out. The - most common example is the branch being rebased is checked out and - decorates the tip commit. If the user is rebasing commits reachable from - a different branch that is checked out in a different worktree, then - they may be surprised to not see that ref update. However, it's probably - best to not optimize for this scenario and do the safest thing that will - result in a successful rebase. A comment is left in the TODO list that - signals that these refs are currently checked out. + This change update the documentation and builtin to accept the + --update-refs option as well as updating the todo file with the + 'update-ref' commands. Tests are added to ensure that these todo + commands are added in the correct locations. + + A future change will update the behavior to actually update the refs + at the end of the rebase sequence. Signed-off-by: Derrick Stolee <derrickstolee@github.com> @@ sequencer.c #include "rebase-interactive.h" #include "reset.h" +#include "branch.h" -+#include "log-tree.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, + size_t items_alloc; + struct strbuf *buf; + struct commit *commit; ++ struct string_list refs_to_oids; +}; + -+static int add_branch_for_decoration(const struct name_decoration *d, void *data) ++static int add_decorations_to_list(const struct commit *commit, ++ struct todo_add_branch_context *ctx) +{ -+ struct todo_add_branch_context *ctx = data; -+ size_t base_offset = ctx->buf->len; -+ struct todo_item *item; -+ char *path; ++ const struct name_decoration *decoration = get_name_decoration(&commit->object); + -+ ALLOC_GROW(ctx->items, -+ ctx->items_nr + 1, -+ ctx->items_alloc); -+ item = &ctx->items[ctx->items_nr]; -+ memset(item, 0, sizeof(*item)); ++ while (decoration) { ++ struct todo_item *item; ++ const char *path; ++ size_t base_offset = ctx->buf->len; + -+ /* If the branch is checked out, then leave a comment instead. */ -+ if (branch_checked_out(d->name, &path)) { -+ item->command = TODO_COMMENT; -+ strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", -+ d->name, path); -+ free(path); -+ } else { -+ item->command = TODO_LABEL; -+ strbuf_addf(ctx->buf, "for-update-refs/%s\n", d->name); -+ } ++ ALLOC_GROW(ctx->items, ++ ctx->items_nr + 1, ++ ctx->items_alloc); ++ item = &ctx->items[ctx->items_nr]; ++ memset(item, 0, sizeof(*item)); ++ ++ /* If the branch is checked out, then leave a comment instead. */ ++ if ((path = branch_checked_out(decoration->name))) { ++ item->command = TODO_COMMENT; ++ strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", ++ decoration->name, path); ++ } else { ++ struct string_list_item *sti; ++ item->command = TODO_UPDATE_REF; ++ strbuf_addf(ctx->buf, "%s\n", decoration->name); ++ ++ sti = string_list_append(&ctx->refs_to_oids, ++ decoration->name); ++ sti->util = oiddup(the_hash_algo->null_oid); ++ } ++ ++ item->offset_in_buf = base_offset; ++ item->arg_offset = base_offset; ++ item->arg_len = ctx->buf->len - base_offset; ++ ctx->items_nr++; + -+ item->offset_in_buf = base_offset; -+ item->arg_offset = base_offset; -+ item->arg_len = ctx->buf->len - base_offset; -+ ctx->items_nr++; ++ decoration = decoration->next; ++ } + + return 0; +} @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, + struct decoration_filter decoration_filter = { + .include_ref_pattern = &decorate_refs_include, + .exclude_ref_pattern = &decorate_refs_exclude, -+ .exclude_ref_config_pattern = &decorate_refs_exclude_config ++ .exclude_ref_config_pattern = &decorate_refs_exclude_config, + }; + struct todo_add_branch_context ctx = { + .buf = &todo_list->buf, ++ .refs_to_oids = STRING_LIST_INIT_DUP, + }; + + ctx.items_alloc = 2 * todo_list->nr + 1; @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, + + if (item->commit) { + ctx.commit = item->commit; -+ for_each_decoration(item->commit, -+ add_branch_for_decoration, -+ &ctx); ++ add_decorations_to_list(item->commit, &ctx); + } + } + -+ /* Add the "update-refs" step. */ -+ ALLOC_GROW(ctx.items, -+ ctx.items_nr + 1, -+ ctx.items_alloc); -+ memset(&ctx.items[ctx.items_nr], 0, sizeof(struct todo_item)); -+ ctx.items[ctx.items_nr].command = TODO_UPDATE_REFS; -+ ctx.items_nr++; -+ ++ string_list_clear(&ctx.refs_to_oids, 1); + free(todo_list->items); + todo_list->items = ctx.items; + todo_list->nr = ctx.items_nr; @@ sequencer.h: int complete_action(struct repository *r, struct replay_opts *opts, int todo_list_rearrange_squash(struct todo_list *todo_list); + ## t/t2407-worktree-heads.sh ## +@@ t/t2407-worktree-heads.sh: test_expect_success 'refuse to overwrite when in error states' ' + done + ' + ++. "$TEST_DIRECTORY"/lib-rebase.sh ++ ++test_expect_success !SANITIZE_LEAK 'refuse to overwrite during rebase with --update-refs' ' ++ git commit --fixup HEAD~2 --allow-empty && ++ ( ++ set_cat_todo_editor && ++ test_must_fail git rebase -i --update-refs HEAD~3 >todo && ++ ! grep "update-refs" todo ++ ) && ++ git branch -f allow-update HEAD~2 && ++ ( ++ set_cat_todo_editor && ++ test_must_fail git rebase -i --update-refs HEAD~3 >todo && ++ grep "update-ref refs/heads/allow-update" todo ++ ) ++' ++ + test_done + ## t/t3404-rebase-interactive.sh ## @@ t/t3404-rebase-interactive.sh: test_expect_success 'ORIG_HEAD is updated correctly' ' test_cmp_rev ORIG_HEAD test-orig-head@{1} @@ t/t3404-rebase-interactive.sh: test_expect_success 'ORIG_HEAD is updated correct + + cat >expect <<-EOF && + pick $(git log -1 --format=%h J) J -+ label for-update-refs/refs/heads/second -+ label for-update-refs/refs/heads/first ++ update-ref refs/heads/second ++ update-ref refs/heads/first + pick $(git log -1 --format=%h K) K + pick $(git log -1 --format=%h L) L + fixup $(git log -1 --format=%h update-refs) fixup! L # empty -+ label for-update-refs/refs/heads/third ++ update-ref refs/heads/third + pick $(git log -1 --format=%h M) M -+ label for-update-refs/refs/heads/no-conflict-branch -+ label for-update-refs/refs/heads/shared-tip -+ update-refs ++ update-ref refs/heads/no-conflict-branch ++ update-ref refs/heads/shared-tip + EOF + + test_must_fail git rebase -i --autosquash --update-refs primary >todo && @@ t/t3404-rebase-interactive.sh: test_expect_success 'ORIG_HEAD is updated correct + reset onto + pick $(git log -1 --format=%h branch2~1) F + pick $(git log -1 --format=%h branch2) I -+ label for-update-refs/refs/heads/branch2 ++ update-ref refs/heads/branch2 + label merge + reset onto + pick $(git log -1 --format=%h refs/heads/second) J -+ label for-update-refs/refs/heads/second -+ label for-update-refs/refs/heads/first ++ update-ref refs/heads/second ++ update-ref refs/heads/first + pick $(git log -1 --format=%h refs/heads/third~1) K + pick $(git log -1 --format=%h refs/heads/third) L + fixup $(git log -1 --format=%h update-refs-with-merge) fixup! L # empty -+ label for-update-refs/refs/heads/third ++ update-ref refs/heads/third + pick $(git log -1 --format=%h HEAD~2) M -+ label for-update-refs/refs/heads/no-conflict-branch ++ update-ref refs/heads/no-conflict-branch + merge -C $(git log -1 --format=%h HEAD~1) merge # merge -+ label for-update-refs/refs/heads/merge-branch -+ update-refs ++ update-ref refs/heads/merge-branch + EOF + + test_must_fail git rebase -i --autosquash \ 6: 68f8e51b19c ! 7: 72e0481b643 sequencer: implement 'update-refs' command @@ Metadata Author: Derrick Stolee <derrickstolee@github.com> ## Commit message ## - sequencer: implement 'update-refs' command + rebase: update refs from 'update-ref' commands - The previous change allowed 'git rebase --update-refs' to create 'label' - commands for each branch among the commits being rewritten and add an - 'update-refs' command at the end of the todo list. Now, teach Git to - update the refs during that final 'update-refs' command. + The previous change introduced the 'git rebase --update-refs' option + which added 'update-ref <ref>' commands to the todo list of an + interactive rebase. - We need to create an array of new and old OIDs for each ref by iterating - over the refs/rewritten/for-update-refs/ namespace. We cannot update the - refs in-place since this will confuse the refs iterator. + Teach Git to record the HEAD position when reaching these 'update-ref' + commands. The ref/OID pair is stored in the + $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this + file to avoid having other processes updating the refs in that file + while the rebase is in progress. + + Not only do we update the file when the sequencer reaches these + 'update-ref' commands, we then update the refs themselves at the end of + the rebase sequence. If the rebase is aborted before this final step, + then the refs are not updated. Signed-off-by: Derrick Stolee <derrickstolee@github.com> ## sequencer.c ## +@@ + #include "rebase-interactive.h" + #include "reset.h" + #include "branch.h" ++#include "log-tree.h" + + #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" + +@@ sequencer.c: static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") + */ + static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") + ++/* ++ * The update-refs file stores a list of refs that will be updated at the end ++ * of the rebase sequence. The 'update-ref <ref>' commands in the todo file ++ * update the OIDs for the refs in this file, but the refs are not updated ++ * until the end of the rebase sequence. ++ */ ++static GIT_PATH_FUNC(rebase_path_update_refs, "rebase-merge/update-refs") ++ + /* + * The following files are written by git-rebase just after parsing the + * command-line. @@ sequencer.c: leave_merge: return ret; } --static int do_update_refs(struct repository *r) -+struct update_refs_context { -+ struct ref_store *refs; -+ char **ref_names; -+ struct object_id *old; -+ struct object_id *new; -+ size_t nr; -+ size_t alloc; -+}; -+ -+static int add_ref_to_context(const char *refname, -+ const struct object_id *oid, -+ int flags, -+ void *data) +-static int do_update_ref(struct repository *r, const char *ref_name) ++static int write_update_refs_state(struct string_list *refs_to_oids) ++{ ++ int result = 0; ++ FILE *fp = NULL; ++ struct string_list_item *item; ++ char *path = xstrdup(rebase_path_update_refs()); ++ ++ if (safe_create_leading_directories(path)) { ++ result = error(_("unable to create leading directories of %s"), ++ path); ++ goto cleanup; ++ } ++ ++ fp = fopen(path, "w"); ++ if (!fp) { ++ result = error_errno(_("could not open '%s' for writing"), path); ++ goto cleanup; ++ } ++ ++ for_each_string_list_item(item, refs_to_oids) ++ fprintf(fp, "%s\n%s\n", item->string, oid_to_hex(item->util)); ++ ++cleanup: ++ if (fp) ++ fclose(fp); ++ return result; ++} ++ ++static int do_update_ref(struct repository *r, const char *refname) { -+ int f = 0; -+ const char *name; -+ struct update_refs_context *ctx = data; ++ struct string_list_item *item; ++ struct string_list list = STRING_LIST_INIT_DUP; ++ int found = 0; ++ ++ sequencer_get_update_refs_state(r->gitdir, &list); + -+ ALLOC_GROW(ctx->ref_names, ctx->nr + 1, ctx->alloc); -+ ALLOC_GROW(ctx->old, ctx->nr + 1, ctx->alloc); -+ ALLOC_GROW(ctx->new, ctx->nr + 1, ctx->alloc); ++ for_each_string_list_item(item, &list) { ++ if (!strcmp(item->string, refname)) { ++ struct object_id oid; ++ free(item->util); ++ found = 1; + -+ if (!skip_prefix(refname, "refs/rewritten/for-update-refs/", &name)) -+ return 1; ++ if (!read_ref("HEAD", &oid)) { ++ item->util = oiddup(&oid); ++ break; ++ } ++ } ++ } + -+ ctx->ref_names[ctx->nr] = xstrdup(name); -+ oidcpy(&ctx->new[ctx->nr], oid); -+ if (!refs_resolve_ref_unsafe(ctx->refs, name, 0, -+ &ctx->old[ctx->nr], &f)) -+ return 1; ++ if (!found) { ++ struct object_id oid; ++ item = string_list_append(&list, refname); + -+ ctx->nr++; ++ if (!read_ref("HEAD", &oid)) ++ item->util = oiddup(&oid); ++ else ++ item->util = oiddup(the_hash_algo->null_oid); ++ } ++ ++ write_update_refs_state(&list); ++ string_list_clear(&list, 1); return 0; } +static int do_update_refs(struct repository *r) +{ -+ int i, res; -+ struct update_refs_context ctx = { -+ .refs = get_main_ref_store(r), -+ .alloc = 16, -+ }; -+ ALLOC_ARRAY(ctx.ref_names, ctx.alloc); -+ ALLOC_ARRAY(ctx.old, ctx.alloc); -+ ALLOC_ARRAY(ctx.new, ctx.alloc); -+ -+ res = refs_for_each_fullref_in(ctx.refs, -+ "refs/rewritten/for-update-refs/", -+ add_ref_to_context, -+ &ctx); -+ -+ for (i = 0; !res && i < ctx.nr; i++) -+ res = refs_update_ref(ctx.refs, "rewritten during rebase", -+ ctx.ref_names[i], -+ &ctx.new[i], &ctx.old[i], ++ int res = 0; ++ struct string_list_item *item; ++ struct string_list refs_to_oids = STRING_LIST_INIT_DUP; ++ struct ref_store *refs = get_main_ref_store(r); ++ ++ sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); ++ ++ for_each_string_list_item(item, &refs_to_oids) { ++ struct object_id *oid_to = item->util; ++ struct object_id oid_from; ++ ++ if (oideq(oid_to, the_hash_algo->null_oid)) { ++ /* ++ * Ref was not updated. User may have deleted the ++ * 'update-ref' step. ++ */ ++ continue; ++ } ++ ++ if (read_ref(item->string, &oid_from)) { ++ /* ++ * The ref does not exist. The user probably ++ * inserted a new 'update-ref' step with a new ++ * branch name. ++ */ ++ oidcpy(&oid_from, the_hash_algo->null_oid); ++ } ++ ++ res |= refs_update_ref(refs, "rewritten during rebase", ++ item->string, ++ oid_to, &oid_from, + 0, UPDATE_REFS_MSG_ON_ERR); ++ } + -+ for (i = 0; i < ctx.nr; i++) -+ free(ctx.ref_names[i]); -+ free(ctx.ref_names); -+ free(ctx.old); -+ free(ctx.new); ++ string_list_clear(&refs_to_oids, 1); + return res; +} + static int is_final_fixup(struct todo_list *todo_list) { int i = todo_list->current; +@@ sequencer.c: cleanup_head_ref: + strbuf_release(&head_ref); + } + ++ do_update_refs(r); ++ + /* + * Sequence of picks finished successfully; cleanup by + * removing the .git/sequencer directory +@@ sequencer.c: static int todo_list_add_update_ref_commands(struct todo_list *todo_list) + } + } + ++ write_update_refs_state(&ctx.refs_to_oids); ++ + string_list_clear(&ctx.refs_to_oids, 1); + free(todo_list->items); + todo_list->items = ctx.items; ## t/t3404-rebase-interactive.sh ## @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds commands with --rebase-merges' ' @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds commands + git branch -f third HEAD~1 && + test_commit extra2 fileX && + git commit --amend --fixup=L && -+ ( -+ git rebase -i --autosquash --update-refs primary && -+ -+ compare_two_refs HEAD~3 refs/heads/first && -+ compare_two_refs HEAD~3 refs/heads/second && -+ compare_two_refs HEAD~1 refs/heads/third && -+ compare_two_refs HEAD refs/heads/no-conflict-branch -+ ) ++ ++ git rebase -i --autosquash --update-refs primary && ++ ++ compare_two_refs HEAD~3 refs/heads/first && ++ compare_two_refs HEAD~3 refs/heads/second && ++ compare_two_refs HEAD~1 refs/heads/third && ++ compare_two_refs HEAD refs/heads/no-conflict-branch +' + # This must be the last test in this file 7: 3d7d3f656b4 ! 8: d2cfdbfc431 rebase: add rebase.updateRefs config option @@ Documentation/git-rebase.txt: start would be overridden by the presence of or point to a `squash! ...` or `fixup! ...` commit are not updated in this way. ++ -+If the `--update-refs` option is enabled by default using the -+configuration variable `rebase.updateRefs`, this option can be -+used to override and disable this setting. ++If the configuration variable `rebase.updateRefs` is set, then this option ++can be used to override and disable this setting. INCOMPATIBLE OPTIONS -------------------- -- gitgitgadget ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v3 1/8] t2407: test branches currently using apply backend 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 ` Derrick Stolee via GitGitGadget 2022-06-28 20:44 ` Junio C Hamano 2022-06-28 13:25 ` [PATCH v3 2/8] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget ` (9 subsequent siblings) 10 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The tests in t2407 that verify the branch_checked_out() helper in the case of bisects and rebases were added by 9347303db89 (branch: check for bisects and rebases, 2022-06-08). However, that commit failed to check for rebases that are using the 'apply' backend. Add a test that checks the apply backend. The implementation was already correct here, but it is good to have regression tests before modifying the implementation further. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- t/t2407-worktree-heads.sh | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index b6be42f74a2..4f59bc21303 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -54,7 +54,18 @@ test_expect_success 'refuse to overwrite: worktree in bisect' ' grep "cannot force update the branch '\''fake-2'\'' checked out at.*wt-4" err ' -test_expect_success 'refuse to overwrite: worktree in rebase' ' +test_expect_success 'refuse to overwrite: worktree in rebase (apply)' ' + test_when_finished rm -rf .git/worktrees/wt-*/rebase-apply && + + mkdir -p .git/worktrees/wt-3/rebase-apply && + echo refs/heads/fake-1 >.git/worktrees/wt-3/rebase-apply/head-name && + echo refs/heads/fake-2 >.git/worktrees/wt-3/rebase-apply/onto && + + test_must_fail git branch -f fake-1 HEAD 2>err && + grep "cannot force update the branch '\''fake-1'\'' checked out at.*wt-3" err +' + +test_expect_success 'refuse to overwrite: worktree in rebase (merge)' ' test_when_finished rm -rf .git/worktrees/wt-*/rebase-merge && mkdir -p .git/worktrees/wt-3/rebase-merge && -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v3 1/8] t2407: test branches currently using apply backend 2022-06-28 13:25 ` [PATCH v3 1/8] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget @ 2022-06-28 20:44 ` Junio C Hamano 2022-06-29 12:54 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-06-28 20:44 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Derrick Stolee <derrickstolee@github.com> > > The tests in t2407 that verify the branch_checked_out() helper in the > case of bisects and rebases were added by 9347303db89 (branch: check for > bisects and rebases, 2022-06-08). However, that commit failed to check > for rebases that are using the 'apply' backend. > > Add a test that checks the apply backend. The implementation was already > correct here, but it is good to have regression tests before modifying > the implementation further. > > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > t/t2407-worktree-heads.sh | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh > index b6be42f74a2..4f59bc21303 100755 > --- a/t/t2407-worktree-heads.sh > +++ b/t/t2407-worktree-heads.sh > @@ -54,7 +54,18 @@ test_expect_success 'refuse to overwrite: worktree in bisect' ' > grep "cannot force update the branch '\''fake-2'\'' checked out at.*wt-4" err > ' > > -test_expect_success 'refuse to overwrite: worktree in rebase' ' > +test_expect_success 'refuse to overwrite: worktree in rebase (apply)' ' > + test_when_finished rm -rf .git/worktrees/wt-*/rebase-apply && > + > + mkdir -p .git/worktrees/wt-3/rebase-apply && > + echo refs/heads/fake-1 >.git/worktrees/wt-3/rebase-apply/head-name && > + echo refs/heads/fake-2 >.git/worktrees/wt-3/rebase-apply/onto && > + > + test_must_fail git branch -f fake-1 HEAD 2>err && > + grep "cannot force update the branch '\''fake-1'\'' checked out at.*wt-3" err > +' > + > +test_expect_success 'refuse to overwrite: worktree in rebase (merge)' ' > test_when_finished rm -rf .git/worktrees/wt-*/rebase-merge && > > mkdir -p .git/worktrees/wt-3/rebase-merge && This is not the first offender, since the other existing one is doing the same, but it is a bit sad that this makes it worse to expose and depend on the details of the way the rebase happens to be currently implemented. Perhaps a more kosher way to do this is to find an commit that surely would not allow fake-1 branch to be cleanly rebased onto and actually start (and cause it to stop) a rebase. I notice that the original offence was committed fairly recently, by d2ba271a (branch: check for bisects and rebases, 2022-06-14) that we can easily eject out of the 'next' branch when we rewind and rebuild it, if we wanted to. Anyway, let's read on. Thanks. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 1/8] t2407: test branches currently using apply backend 2022-06-28 20:44 ` Junio C Hamano @ 2022-06-29 12:54 ` Derrick Stolee 2022-06-30 16:44 ` Junio C Hamano 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-06-29 12:54 UTC (permalink / raw) To: Junio C Hamano, Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 6/28/22 4:44 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: >> -test_expect_success 'refuse to overwrite: worktree in rebase' ' >> +test_expect_success 'refuse to overwrite: worktree in rebase (apply)' ' >> + test_when_finished rm -rf .git/worktrees/wt-*/rebase-apply && >> + >> + mkdir -p .git/worktrees/wt-3/rebase-apply && >> + echo refs/heads/fake-1 >.git/worktrees/wt-3/rebase-apply/head-name && >> + echo refs/heads/fake-2 >.git/worktrees/wt-3/rebase-apply/onto && >> + >> + test_must_fail git branch -f fake-1 HEAD 2>err && >> + grep "cannot force update the branch '\''fake-1'\'' checked out at.*wt-3" err >> +' >> + >> +test_expect_success 'refuse to overwrite: worktree in rebase (merge)' ' >> test_when_finished rm -rf .git/worktrees/wt-*/rebase-merge && >> >> mkdir -p .git/worktrees/wt-3/rebase-merge && > > This is not the first offender, since the other existing one is > doing the same, but it is a bit sad that this makes it worse to > expose and depend on the details of the way the rebase happens to be > currently implemented. This is true that it is based on how it's currently implemented. I was confused that none of these rebase storage mechanisms are documented anywhere (so I couldn't extend that documentation with the new update-refs file added in this series). But perhaps that's by design: this is intentionally an internal implementation detail. I do wonder where that leaves third-party implementations like libgit2 to handle any changes we make in this space, or how much we care about supporting someone rebasing in a worktree with an older Git client while also checking out a branch in another worktree using a newer one. > Perhaps a more kosher way to do this is to find an commit that > surely would not allow fake-1 branch to be cleanly rebased onto and > actually start (and cause it to stop) a rebase. This, and the equivalent "pause in the middle of a bisect" would be ways to hide these internal details. While I was thinking that this provides a low-weight mechanism for testing the implementation of branch_checked_out(), it is only testing the implementation but not the _intention_. If 'git rebase' changed its backend, then these tests would not start failing as we need them to. > I notice that the original offence was committed fairly recently, by > d2ba271a (branch: check for bisects and rebases, 2022-06-14) that we > can easily eject out of the 'next' branch when we rewind and rebuild > it, if we wanted to. I can either re-roll that series or create a new forward-fix that includes the functionality of this test. Both are the same amount of work for me, so let me know which you prefer. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 1/8] t2407: test branches currently using apply backend 2022-06-29 12:54 ` Derrick Stolee @ 2022-06-30 16:44 ` Junio C Hamano 2022-06-30 17:35 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-06-30 16:44 UTC (permalink / raw) To: Derrick Stolee Cc: Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren Derrick Stolee <derrickstolee@github.com> writes: > I can either re-roll that series or create a new forward-fix that > includes the functionality of this test. Both are the same amount of > work for me, so let me know which you prefer. Either is fine by me. Other than this small glitch in the test, the remainder of the "should we allow this branch to be touched?" topic looked really cleanly done and ready to be unleashed to the public. Thanks. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 1/8] t2407: test branches currently using apply backend 2022-06-30 16:44 ` Junio C Hamano @ 2022-06-30 17:35 ` Derrick Stolee 0 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee @ 2022-06-30 17:35 UTC (permalink / raw) To: Junio C Hamano Cc: Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 6/30/2022 12:44 PM, Junio C Hamano wrote: > Derrick Stolee <derrickstolee@github.com> writes: > >> I can either re-roll that series or create a new forward-fix that >> includes the functionality of this test. Both are the same amount of >> work for me, so let me know which you prefer. > > Either is fine by me. Other than this small glitch in the test, the > remainder of the "should we allow this branch to be touched?" topic > looked really cleanly done and ready to be unleashed to the public. If you don't mind, I've got some forward-fixes prepped for the next version of this topic. They are test-only changes and do not change the implementation of branch_checked_out(). One thing I noticed from looking back on the series is that I wrote a test that specifically tests a case that is impossible to construct without modifying the .git directory directly (having two worktrees "reserving" a branch for different reasons). That test will need to stay as one that knows about the internals of this storage, but the others can be replaced with more opaque steps. In patch 2 of this series, I test the 'update-refs' storage using a similar mechanism. I can replace that with the opaque version that calls 'git rebase --update-refs', but only as part of patch 7, since writing the file is not implemented yet. I was going to ask if I should reorder the patches, but there is a possibility that we will change the storage of these "reserved" refs yet again, so I won't focus on this point quite yet. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v3 2/8] branch: consider refs under 'update-refs' 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget 2022-06-28 13:25 ` [PATCH v3 1/8] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 ` Derrick Stolee via GitGitGadget 2022-06-28 20:48 ` Junio C Hamano 2022-06-30 9:32 ` Phillip Wood 2022-06-28 13:25 ` [PATCH v3 3/8] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget ` (8 subsequent siblings) 10 siblings, 2 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The branch_checked_out() helper helps commands like 'git branch' and 'git fetch' from overwriting refs that are currently checked out in other worktrees. A future update to 'git rebase' will introduce a new '--update-refs' option which will update the local refs that point to commits that are being rebased. To avoid collisions as the rebase completes, we want to make the future data store for these refs to be considered by branch_checked_out(). The data store is a plaintext file inside the 'rebase-merge' directory for that worktree. The file alternates refnames and OIDs. The OIDs will be used to store the to-be-written values as the rebase progresses, but can be ignored at the moment. Create a new sequencer_get_update_refs_state() method that parses this file and populates a struct string_list with the ref-OID pairs. We can then use this list to add to the current_checked_out_branches strmap used by branch_checked_out(). To properly navigate to the rebase directory for a given worktree, extract the static strbuf_worktree_gitdir() method to a public API method. We can test that this works without having Git write this file by artificially creating one in our test script. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- branch.c | 13 ++++++++++++ sequencer.c | 42 +++++++++++++++++++++++++++++++++++++++ sequencer.h | 9 +++++++++ t/t2407-worktree-heads.sh | 22 ++++++++++++++++---- 4 files changed, 82 insertions(+), 4 deletions(-) diff --git a/branch.c b/branch.c index 526e8237673..f252c4eef6c 100644 --- a/branch.c +++ b/branch.c @@ -365,6 +365,7 @@ static void prepare_checked_out_branches(void) char *old; struct wt_status_state state = { 0 }; struct worktree *wt = worktrees[i++]; + struct string_list update_refs = STRING_LIST_INIT_DUP; if (wt->is_bare) continue; @@ -400,6 +401,18 @@ static void prepare_checked_out_branches(void) strbuf_release(&ref); } wt_status_state_free_buffers(&state); + + if (!sequencer_get_update_refs_state(get_worktree_git_dir(wt), + &update_refs)) { + struct string_list_item *item; + for_each_string_list_item(item, &update_refs) { + old = strmap_put(¤t_checked_out_branches, + item->string, + xstrdup(wt->path)); + free(old); + } + string_list_clear(&update_refs, 1); + } } free_worktrees(worktrees); diff --git a/sequencer.c b/sequencer.c index 8c3ed3532ac..1094e146b99 100644 --- a/sequencer.c +++ b/sequencer.c @@ -5901,3 +5901,45 @@ int sequencer_determine_whence(struct repository *r, enum commit_whence *whence) return 0; } + +int sequencer_get_update_refs_state(const char *wt_dir, + struct string_list *refs) +{ + int result = 0; + struct stat st; + FILE *fp = NULL; + struct strbuf ref = STRBUF_INIT; + struct strbuf hash = STRBUF_INIT; + char *path = xstrfmt("%s/rebase-merge/update-refs", wt_dir); + + if (stat(path, &st)) + goto cleanup; + + fp = fopen(path, "r"); + if (!fp) + goto cleanup; + + while (strbuf_getline(&ref, fp) != EOF) { + struct object_id oid; + struct string_list_item *item; + + if (strbuf_getline(&hash, fp) == EOF || + get_oid_hex(hash.buf, &oid)) { + warning(_("update-refs file at '%s' is invalid"), + path); + result = -1; + goto cleanup; + } + + item = string_list_append(refs, ref.buf); + item->util = oiddup(&oid); + } + +cleanup: + if (fp) + fclose(fp); + free(path); + strbuf_release(&ref); + strbuf_release(&hash); + return result; +} diff --git a/sequencer.h b/sequencer.h index da64473636b..3ae541bb145 100644 --- a/sequencer.h +++ b/sequencer.h @@ -232,4 +232,13 @@ void sequencer_post_commit_cleanup(struct repository *r, int verbose); int sequencer_get_last_command(struct repository* r, enum replay_action *action); int sequencer_determine_whence(struct repository *r, enum commit_whence *whence); + +/** + * Append the set of ref-OID pairs that are currently stored for the 'git + * rebase --update-refs' feature if such a rebase is currently happening. + * + * Localized to a worktree's git dir. + */ +int sequencer_get_update_refs_state(const char *wt_dir, struct string_list *refs); + #endif /* SEQUENCER_H */ diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index 4f59bc21303..f1e9e172a0c 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -7,8 +7,11 @@ TEST_PASSES_SANITIZE_LEAK=true test_expect_success 'setup' ' test_commit init && - git branch -f fake-1 && - git branch -f fake-2 && + + for i in 1 2 3 4 + do + git branch -f fake-$i || return 1 + done && for i in 1 2 3 4 do @@ -73,8 +76,19 @@ test_expect_success 'refuse to overwrite: worktree in rebase (merge)' ' echo refs/heads/fake-1 >.git/worktrees/wt-3/rebase-merge/head-name && echo refs/heads/fake-2 >.git/worktrees/wt-3/rebase-merge/onto && - test_must_fail git branch -f fake-1 HEAD 2>err && - grep "cannot force update the branch '\''fake-1'\'' checked out at.*wt-3" err + cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && + refs/heads/fake-3 + $(git rev-parse HEAD~1) + refs/heads/fake-4 + $(git rev-parse HEAD) + EOF + + for i in 1 3 4 + do + test_must_fail git branch -f fake-$i HEAD 2>err && + grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || + return 1 + done ' test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: checked out' ' -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v3 2/8] branch: consider refs under 'update-refs' 2022-06-28 13:25 ` [PATCH v3 2/8] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget @ 2022-06-28 20:48 ` Junio C Hamano 2022-06-29 12:58 ` Derrick Stolee 2022-06-30 9:32 ` Phillip Wood 1 sibling, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-06-28 20:48 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Derrick Stolee <derrickstolee@github.com> > > The branch_checked_out() helper helps commands like 'git branch' and > 'git fetch' from overwriting refs that are currently checked out in > other worktrees. > > A future update to 'git rebase' will introduce a new '--update-refs' > option which will update the local refs that point to commits that are > being rebased. To avoid collisions as the rebase completes, we want to > make the future data store for these refs to be considered by > branch_checked_out(). > > The data store is a plaintext file inside the 'rebase-merge' directory > for that worktree. The file alternates refnames and OIDs. The OIDs will > be used to store the to-be-written values as the rebase progresses, but > can be ignored at the moment. > > Create a new sequencer_get_update_refs_state() method that parses this > file and populates a struct string_list with the ref-OID pairs. We can > then use this list to add to the current_checked_out_branches strmap > used by branch_checked_out(). > > To properly navigate to the rebase directory for a given worktree, > extract the static strbuf_worktree_gitdir() method to a public API > method. > > We can test that this works without having Git write this file by > artificially creating one in our test script. Hmph, I am not thrilled to see an ad-hoc text file for things like this. Do the objects that appear in this file otherwise already anchored by existing refs, or can some of them be "floating" without any refs pointing at them, i.e. making the "connected" and "fsck" machinery also being aware of them to protect them from collected as garbage and reported as dangling/unreachable? ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 2/8] branch: consider refs under 'update-refs' 2022-06-28 20:48 ` Junio C Hamano @ 2022-06-29 12:58 ` Derrick Stolee 2022-06-30 9:47 ` Phillip Wood 2022-06-30 16:49 ` Junio C Hamano 0 siblings, 2 replies; 144+ messages in thread From: Derrick Stolee @ 2022-06-29 12:58 UTC (permalink / raw) To: Junio C Hamano, Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 6/28/22 4:48 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > >> From: Derrick Stolee <derrickstolee@github.com> >> >> The branch_checked_out() helper helps commands like 'git branch' and >> 'git fetch' from overwriting refs that are currently checked out in >> other worktrees. >> >> A future update to 'git rebase' will introduce a new '--update-refs' >> option which will update the local refs that point to commits that are >> being rebased. To avoid collisions as the rebase completes, we want to >> make the future data store for these refs to be considered by >> branch_checked_out(). >> >> The data store is a plaintext file inside the 'rebase-merge' directory >> for that worktree. The file alternates refnames and OIDs. The OIDs will >> be used to store the to-be-written values as the rebase progresses, but >> can be ignored at the moment. >> >> Create a new sequencer_get_update_refs_state() method that parses this >> file and populates a struct string_list with the ref-OID pairs. We can >> then use this list to add to the current_checked_out_branches strmap >> used by branch_checked_out(). >> >> To properly navigate to the rebase directory for a given worktree, >> extract the static strbuf_worktree_gitdir() method to a public API >> method. >> >> We can test that this works without having Git write this file by >> artificially creating one in our test script. > > Hmph, I am not thrilled to see an ad-hoc text file for things like > this. Do the objects that appear in this file otherwise already > anchored by existing refs, or can some of them be "floating" without > any refs pointing at them, i.e. making the "connected" and "fsck" > machinery also being aware of them to protect them from collected as > garbage and reported as dangling/unreachable? You're right that we could have this sequence of events in a todo file: pick deadbeef Here is a commit that will become unreachable update-ref will-be-lost squash 1234567 make will-be-lost unreachable ... So if a GC runs after this 'update-ref' step but before the rebase completes, then that commit can be lost. The ref-update at the end of the rebase process would fail. I wonder how important such a situation is, though. But I'm willing to add the extra lookups in 'git gc' and 'git fsck' to make this a non-issue. I'll take a look to see where the other refs in rebase-<backend>/* are handled by these processes. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 2/8] branch: consider refs under 'update-refs' 2022-06-29 12:58 ` Derrick Stolee @ 2022-06-30 9:47 ` Phillip Wood 2022-06-30 16:50 ` Junio C Hamano 2022-06-30 16:49 ` Junio C Hamano 1 sibling, 1 reply; 144+ messages in thread From: Phillip Wood @ 2022-06-30 9:47 UTC (permalink / raw) To: Derrick Stolee, Junio C Hamano, Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Elijah Newren On 29/06/2022 13:58, Derrick Stolee wrote: > On 6/28/22 4:48 PM, Junio C Hamano wrote: >> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: >> >>> From: Derrick Stolee <derrickstolee@github.com> >>> >>> The branch_checked_out() helper helps commands like 'git branch' and >>> 'git fetch' from overwriting refs that are currently checked out in >>> other worktrees. >>> >>> A future update to 'git rebase' will introduce a new '--update-refs' >>> option which will update the local refs that point to commits that are >>> being rebased. To avoid collisions as the rebase completes, we want to >>> make the future data store for these refs to be considered by >>> branch_checked_out(). >>> >>> The data store is a plaintext file inside the 'rebase-merge' directory >>> for that worktree. The file alternates refnames and OIDs. The OIDs will >>> be used to store the to-be-written values as the rebase progresses, but >>> can be ignored at the moment. >>> >>> Create a new sequencer_get_update_refs_state() method that parses this >>> file and populates a struct string_list with the ref-OID pairs. We can >>> then use this list to add to the current_checked_out_branches strmap >>> used by branch_checked_out(). >>> >>> To properly navigate to the rebase directory for a given worktree, >>> extract the static strbuf_worktree_gitdir() method to a public API >>> method. >>> >>> We can test that this works without having Git write this file by >>> artificially creating one in our test script. >> >> Hmph, I am not thrilled to see an ad-hoc text file for things like >> this. Do the objects that appear in this file otherwise already >> anchored by existing refs, or can some of them be "floating" without >> any refs pointing at them, i.e. making the "connected" and "fsck" >> machinery also being aware of them to protect them from collected as >> garbage and reported as dangling/unreachable? > > You're right that we could have this sequence of events in a todo > file: > > pick deadbeef Here is a commit that will become unreachable > update-ref will-be-lost > > squash 1234567 make will-be-lost unreachable > ... > > So if a GC runs after this 'update-ref' step but before the rebase > completes, then that commit can be lost. The ref-update at the end > of the rebase process would fail. The commit is protected by HEAD's reflog for a while because we update HEAD after each pick but it is not permanently safe. I've taken a look at the first few patches and they're looking good, I hope to look at the rest tomorrow Best Wishes Phillip > I wonder how important such a situation is, though. But I'm willing > to add the extra lookups in 'git gc' and 'git fsck' to make this a > non-issue. > > I'll take a look to see where the other refs in rebase-<backend>/* > are handled by these processes. > > Thanks, > -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 2/8] branch: consider refs under 'update-refs' 2022-06-30 9:47 ` Phillip Wood @ 2022-06-30 16:50 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-30 16:50 UTC (permalink / raw) To: Phillip Wood Cc: Derrick Stolee, Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Elijah Newren Phillip Wood <phillip.wood123@gmail.com> writes: > The commit is protected by HEAD's reflog for a while because we update > HEAD after each pick but it is not permanently safe. Ah, I forgot about that one. Thanks for pointing it out. It does sound like HEAD's reflog makes it safe enough. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 2/8] branch: consider refs under 'update-refs' 2022-06-29 12:58 ` Derrick Stolee 2022-06-30 9:47 ` Phillip Wood @ 2022-06-30 16:49 ` Junio C Hamano 1 sibling, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-30 16:49 UTC (permalink / raw) To: Derrick Stolee Cc: Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren Derrick Stolee <derrickstolee@github.com> writes: > I wonder how important such a situation is, though. But I'm willing > to add the extra lookups in 'git gc' and 'git fsck' to make this a > non-issue. What I was hinting at was if some of this can be done by using a ref, so that we do not have to touch "gc" or "fsck" at all. As to the importance, I would say it is about the same importance as being able to prevent the branches involved in the rebase from touched in other worktrees. We expect it to take sufficiently large wallclock time from the beginning of a rebase and its finish to make such a protection necessary, so an auto-gc may even be started from other worktrees without our knowledge. Thanks. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 2/8] branch: consider refs under 'update-refs' 2022-06-28 13:25 ` [PATCH v3 2/8] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget 2022-06-28 20:48 ` Junio C Hamano @ 2022-06-30 9:32 ` Phillip Wood 2022-06-30 13:35 ` Derrick Stolee 1 sibling, 1 reply; 144+ messages in thread From: Phillip Wood @ 2022-06-30 9:32 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee On 28/06/2022 14:25, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > The branch_checked_out() helper helps commands like 'git branch' and > 'git fetch' from overwriting refs that are currently checked out in > other worktrees. > > A future update to 'git rebase' will introduce a new '--update-refs' > option which will update the local refs that point to commits that are > being rebased. To avoid collisions as the rebase completes, we want to > make the future data store for these refs to be considered by > branch_checked_out(). > > The data store is a plaintext file inside the 'rebase-merge' directory > for that worktree. The file alternates refnames and OIDs. The OIDs will > be used to store the to-be-written values as the rebase progresses, but > can be ignored at the moment. > > Create a new sequencer_get_update_refs_state() method that parses this > file and populates a struct string_list with the ref-OID pairs. We can > then use this list to add to the current_checked_out_branches strmap > used by branch_checked_out(). > > To properly navigate to the rebase directory for a given worktree, > extract the static strbuf_worktree_gitdir() method to a public API > method. > > We can test that this works without having Git write this file by > artificially creating one in our test script. This looks good apart from a couple of small nits > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > branch.c | 13 ++++++++++++ > sequencer.c | 42 +++++++++++++++++++++++++++++++++++++++ > sequencer.h | 9 +++++++++ > t/t2407-worktree-heads.sh | 22 ++++++++++++++++---- > 4 files changed, 82 insertions(+), 4 deletions(-) > > diff --git a/branch.c b/branch.c > index 526e8237673..f252c4eef6c 100644 > --- a/branch.c > +++ b/branch.c > @@ -365,6 +365,7 @@ static void prepare_checked_out_branches(void) > char *old; > struct wt_status_state state = { 0 }; > struct worktree *wt = worktrees[i++]; > + struct string_list update_refs = STRING_LIST_INIT_DUP; > > if (wt->is_bare) > continue; > @@ -400,6 +401,18 @@ static void prepare_checked_out_branches(void) > strbuf_release(&ref); > } > wt_status_state_free_buffers(&state); > + > + if (!sequencer_get_update_refs_state(get_worktree_git_dir(wt), > + &update_refs)) { > + struct string_list_item *item; > + for_each_string_list_item(item, &update_refs) { > + old = strmap_put(¤t_checked_out_branches, > + item->string, > + xstrdup(wt->path)); > + free(old); > + } > + string_list_clear(&update_refs, 1); > + } > } > > free_worktrees(worktrees); > diff --git a/sequencer.c b/sequencer.c > index 8c3ed3532ac..1094e146b99 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -5901,3 +5901,45 @@ int sequencer_determine_whence(struct repository *r, enum commit_whence *whence) > > return 0; > } > + > +int sequencer_get_update_refs_state(const char *wt_dir, > + struct string_list *refs) > +{ > + int result = 0; > + struct stat st; > + FILE *fp = NULL; > + struct strbuf ref = STRBUF_INIT; > + struct strbuf hash = STRBUF_INIT; > + char *path = xstrfmt("%s/rebase-merge/update-refs", wt_dir); I think it would make sense to introduce rebase_path_update_refs() in this patch rather than later in the series > + > + if (stat(path, &st)) > + goto cleanup; What's the reason for stating the file first, rather than just letting fopen() fail if it does not exist? Best Wishes Phillip ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 2/8] branch: consider refs under 'update-refs' 2022-06-30 9:32 ` Phillip Wood @ 2022-06-30 13:35 ` Derrick Stolee 2022-07-01 13:40 ` Phillip Wood 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-06-30 13:35 UTC (permalink / raw) To: phillip.wood, Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren On 6/30/2022 5:32 AM, Phillip Wood wrote: > On 28/06/2022 14:25, Derrick Stolee via GitGitGadget wrote: >> From: Derrick Stolee <derrickstolee@github.com> >> +int sequencer_get_update_refs_state(const char *wt_dir, >> + struct string_list *refs) >> +{ >> + int result = 0; >> + struct stat st; >> + FILE *fp = NULL; >> + struct strbuf ref = STRBUF_INIT; >> + struct strbuf hash = STRBUF_INIT; >> + char *path = xstrfmt("%s/rebase-merge/update-refs", wt_dir); > > I think it would make sense to introduce rebase_path_update_refs() in this patch rather than later in the series The biggest difference is that rebase_path_update_refs() only gives the path for the current worktree, while this method needs to read the file for any worktree. There is likely some opportunity to create rebase_path_update_refs() in a different way that serves both purposes. >> + >> + if (stat(path, &st)) >> + goto cleanup; > > What's the reason for stating the file first, rather than just letting fopen() fail if it does not exist? Not sure what I was looking at that gave this pattern, but you're right that it isn't necessary. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 2/8] branch: consider refs under 'update-refs' 2022-06-30 13:35 ` Derrick Stolee @ 2022-07-01 13:40 ` Phillip Wood 0 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-07-01 13:40 UTC (permalink / raw) To: Derrick Stolee, phillip.wood, Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren Hi Stolee On 30/06/2022 14:35, Derrick Stolee wrote: > On 6/30/2022 5:32 AM, Phillip Wood wrote: >> On 28/06/2022 14:25, Derrick Stolee via GitGitGadget wrote: >>> From: Derrick Stolee <derrickstolee@github.com> > >>> +int sequencer_get_update_refs_state(const char *wt_dir, >>> + struct string_list *refs) >>> +{ >>> + int result = 0; >>> + struct stat st; >>> + FILE *fp = NULL; >>> + struct strbuf ref = STRBUF_INIT; >>> + struct strbuf hash = STRBUF_INIT; >>> + char *path = xstrfmt("%s/rebase-merge/update-refs", wt_dir); >> >> I think it would make sense to introduce rebase_path_update_refs() in this patch rather than later in the series > > The biggest difference is that rebase_path_update_refs() only > gives the path for the current worktree, while this method needs > to read the file for any worktree. That's an important distinction that I'd failed to notice > There is likely some opportunity to create rebase_path_update_refs() > in a different way that serves both purposes. That would be nice, even just having a #define for "rebase-merge/update-refs" and using that here and in the other patch would mean we're not hard coding the filename in two different places. Best Wishes Phillip >>> + >>> + if (stat(path, &st)) >>> + goto cleanup; >> >> What's the reason for stating the file first, rather than just letting fopen() fail if it does not exist? > > Not sure what I was looking at that gave this pattern, but you're > right that it isn't necessary. > > Thanks, > -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v3 3/8] rebase-interactive: update 'merge' description 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget 2022-06-28 13:25 ` [PATCH v3 1/8] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget 2022-06-28 13:25 ` [PATCH v3 2/8] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 ` Derrick Stolee via GitGitGadget 2022-06-28 21:00 ` Junio C Hamano 2022-06-30 9:34 ` Phillip Wood 2022-06-28 13:25 ` [PATCH v3 4/8] sequencer: define array with enum values Derrick Stolee via GitGitGadget ` (7 subsequent siblings) 10 siblings, 2 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The 'merge' command description for the todo list documentation in an interactive rebase has multiple lines. The lines other than the first one start with dots ('.') while the similar multi-line documentation for 'fixup' does not. The 'merge' command was documented when interactive rebase was first ported to C in 145e05ac44b (rebase -i: rewrite append_todo_help() in C, 2018-08-10). These dots might have been carried over from the previous shell implementation. The 'fixup' command was documented more recently in 9e3cebd97cb (rebase -i: add fixup [-C | -c] command, 2021-01-29). Looking at the output in an editor, my personal opinion is that the dots are unnecessary and noisy. Remove them now before adding more commands with multi-line documentation. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- rebase-interactive.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rebase-interactive.c b/rebase-interactive.c index 87649d0c016..22394224faa 100644 --- a/rebase-interactive.c +++ b/rebase-interactive.c @@ -54,9 +54,9 @@ void append_todo_help(int command_count, "l, label <label> = label current HEAD with a name\n" "t, reset <label> = reset HEAD to a label\n" "m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]\n" -". create a merge commit using the original merge commit's\n" -". message (or the oneline, if no original merge commit was\n" -". specified); use -c <commit> to reword the commit message\n" +" create a merge commit using the original merge commit's\n" +" message (or the oneline, if no original merge commit was\n" +" specified); use -c <commit> to reword the commit message\n" "\n" "These lines can be re-ordered; they are executed from top to bottom.\n"); unsigned edit_todo = !(shortrevisions && shortonto); -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v3 3/8] rebase-interactive: update 'merge' description 2022-06-28 13:25 ` [PATCH v3 3/8] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget @ 2022-06-28 21:00 ` Junio C Hamano 2022-06-29 13:02 ` Derrick Stolee 2022-06-30 9:34 ` Phillip Wood 1 sibling, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-06-28 21:00 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Derrick Stolee <derrickstolee@github.com> > > The 'merge' command description for the todo list documentation in an > interactive rebase has multiple lines. The lines other than the first > one start with dots ('.') while the similar multi-line documentation for > 'fixup' does not. > > The 'merge' command was documented when interactive rebase was first > ported to C in 145e05ac44b (rebase -i: rewrite append_todo_help() in C, > 2018-08-10). These dots might have been carried over from the previous > shell implementation. The text indeed does come literally from the block removed by that commit. I wondered if the shell "gettext" in git-i18n.sh had some magic in it, but it does not seem to have anything, and po/git.pot around that time has these lines with leading dots, so I suspect that they were quite deliberately added, not for the reasons of formatting machinery (e.g. preventing somebody in the dataflow from losing leading indentation), but to show them to the end users. Unfortunately, the offending commit 4c68e7dd (sequencer: introduce the `merge` command, 2018-04-25) does not justify them X-<. > Looking at the output in an editor, my personal opinion is that the dots > are unnecessary and noisy. Remove them now before adding more commands > with multi-line documentation. I personally do not mind having them in the UI, but I can also be happy to see them go. It is unlikely that any program is consuming these strings, so I would say this is a fairly safe clean-up. > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > rebase-interactive.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/rebase-interactive.c b/rebase-interactive.c > index 87649d0c016..22394224faa 100644 > --- a/rebase-interactive.c > +++ b/rebase-interactive.c > @@ -54,9 +54,9 @@ void append_todo_help(int command_count, > "l, label <label> = label current HEAD with a name\n" > "t, reset <label> = reset HEAD to a label\n" > "m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]\n" > -". create a merge commit using the original merge commit's\n" > -". message (or the oneline, if no original merge commit was\n" > -". specified); use -c <commit> to reword the commit message\n" > +" create a merge commit using the original merge commit's\n" > +" message (or the oneline, if no original merge commit was\n" > +" specified); use -c <commit> to reword the commit message\n" > "\n" > "These lines can be re-ordered; they are executed from top to bottom.\n"); > unsigned edit_todo = !(shortrevisions && shortonto); ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 3/8] rebase-interactive: update 'merge' description 2022-06-28 21:00 ` Junio C Hamano @ 2022-06-29 13:02 ` Derrick Stolee 2022-06-30 17:05 ` Junio C Hamano 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-06-29 13:02 UTC (permalink / raw) To: Junio C Hamano, Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 6/28/22 5:00 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > >> From: Derrick Stolee <derrickstolee@github.com> >> >> The 'merge' command description for the todo list documentation in an >> interactive rebase has multiple lines. The lines other than the first >> one start with dots ('.') while the similar multi-line documentation for >> 'fixup' does not. >> >> The 'merge' command was documented when interactive rebase was first >> ported to C in 145e05ac44b (rebase -i: rewrite append_todo_help() in C, >> 2018-08-10). These dots might have been carried over from the previous >> shell implementation. > > The text indeed does come literally from the block removed by that > commit. I wondered if the shell "gettext" in git-i18n.sh had some > magic in it, but it does not seem to have anything, and po/git.pot > around that time has these lines with leading dots, so I suspect > that they were quite deliberately added, not for the reasons of > formatting machinery (e.g. preventing somebody in the dataflow from > losing leading indentation), but to show them to the end users. > > Unfortunately, the offending commit 4c68e7dd (sequencer: introduce > the `merge` command, 2018-04-25) does not justify them X-<. > >> Looking at the output in an editor, my personal opinion is that the dots >> are unnecessary and noisy. Remove them now before adding more commands >> with multi-line documentation. > > I personally do not mind having them in the UI, but I can also be > happy to see them go. It is unlikely that any program is consuming > these strings, so I would say this is a fairly safe clean-up. Perhaps I should be a bit clearer that these are appearing in the comment section of the todo-file when presented to the user for editing: # Commands: # p, pick <commit> = use commit # r, reword <commit> = use commit, but edit the commit message # e, edit <commit> = use commit, but stop for amending # s, squash <commit> = use commit, but meld into previous commit # f, fixup [-C | -c] <commit> = like "squash" but keep only the previous # commit's log message, unless -C is used, in which case # keep only this commit's message; -c is same as -C but # opens the editor # x, exec <command> = run command (the rest of the line) using shell # b, break = stop here (continue rebase later with 'git rebase --continue') # d, drop <commit> = remove commit # l, label <label> = label current HEAD with a name # t, reset <label> = reset HEAD to a label # m, merge [-C <commit> | -c <commit>] <label> [# <oneline>] # . create a merge commit using the original merge commit's # . message (or the oneline, if no original merge commit was # . specified); use -c <commit> to reword the commit message # # These lines can be re-ordered; they are executed from top to bottom. # This does not appear to be used anywhere else. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 3/8] rebase-interactive: update 'merge' description 2022-06-29 13:02 ` Derrick Stolee @ 2022-06-30 17:05 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-30 17:05 UTC (permalink / raw) To: Derrick Stolee Cc: Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren Derrick Stolee <derrickstolee@github.com> writes: > Perhaps I should be a bit clearer that these are appearing in the > comment section of the todo-file when presented to the user for editing: Yup, it is good to spell it out for others. > This does not appear to be used anywhere else. Nowhere in our codebase. I do not think we have a way to know if users (e.g. hooks and scripts) depend on them, but I am reasonably sure that nobody would complain ;-) ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 3/8] rebase-interactive: update 'merge' description 2022-06-28 13:25 ` [PATCH v3 3/8] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget 2022-06-28 21:00 ` Junio C Hamano @ 2022-06-30 9:34 ` Phillip Wood 1 sibling, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-06-30 9:34 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee On 28/06/2022 14:25, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > The 'merge' command description for the todo list documentation in an > interactive rebase has multiple lines. The lines other than the first > one start with dots ('.') while the similar multi-line documentation for > 'fixup' does not. > > The 'merge' command was documented when interactive rebase was first > ported to C in 145e05ac44b (rebase -i: rewrite append_todo_help() in C, > 2018-08-10). These dots might have been carried over from the previous > shell implementation. > > The 'fixup' command was documented more recently in 9e3cebd97cb (rebase > -i: add fixup [-C | -c] command, 2021-01-29). > > Looking at the output in an editor, my personal opinion is that the dots > are unnecessary and noisy. Remove them now before adding more commands > with multi-line documentation. I agree, this is a nice cleanup Thanks Phillip > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > rebase-interactive.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/rebase-interactive.c b/rebase-interactive.c > index 87649d0c016..22394224faa 100644 > --- a/rebase-interactive.c > +++ b/rebase-interactive.c > @@ -54,9 +54,9 @@ void append_todo_help(int command_count, > "l, label <label> = label current HEAD with a name\n" > "t, reset <label> = reset HEAD to a label\n" > "m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]\n" > -". create a merge commit using the original merge commit's\n" > -". message (or the oneline, if no original merge commit was\n" > -". specified); use -c <commit> to reword the commit message\n" > +" create a merge commit using the original merge commit's\n" > +" message (or the oneline, if no original merge commit was\n" > +" specified); use -c <commit> to reword the commit message\n" > "\n" > "These lines can be re-ordered; they are executed from top to bottom.\n"); > unsigned edit_todo = !(shortrevisions && shortonto); ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v3 4/8] sequencer: define array with enum values 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget ` (2 preceding siblings ...) 2022-06-28 13:25 ` [PATCH v3 3/8] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 ` Derrick Stolee via GitGitGadget 2022-06-28 21:02 ` Junio C Hamano 2022-06-28 13:25 ` [PATCH v3 5/8] sequencer: add update-ref command Derrick Stolee via GitGitGadget ` (6 subsequent siblings) 10 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The todo_command_info array defines which strings match with which todo_command enum values. The array is defined in the same order as the enum values, but if one changed without the other, then we would have unexpected results. Make it easier to see changes to the enum and this array by using the enum values as the indices of the array. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/sequencer.c b/sequencer.c index 1094e146b99..2ffee79619c 100644 --- a/sequencer.c +++ b/sequencer.c @@ -1685,20 +1685,20 @@ static struct { char c; const char *str; } todo_command_info[] = { - { 'p', "pick" }, - { 0, "revert" }, - { 'e', "edit" }, - { 'r', "reword" }, - { 'f', "fixup" }, - { 's', "squash" }, - { 'x', "exec" }, - { 'b', "break" }, - { 'l', "label" }, - { 't', "reset" }, - { 'm', "merge" }, - { 0, "noop" }, - { 'd', "drop" }, - { 0, NULL } + [TODO_PICK] = { 'p', "pick" }, + [TODO_REVERT] = { 0, "revert" }, + [TODO_EDIT] = { 'e', "edit" }, + [TODO_REWORD] = { 'r', "reword" }, + [TODO_FIXUP] = { 'f', "fixup" }, + [TODO_SQUASH] = { 's', "squash" }, + [TODO_EXEC] = { 'x', "exec" }, + [TODO_BREAK] = { 'b', "break" }, + [TODO_LABEL] = { 'l', "label" }, + [TODO_RESET] = { 't', "reset" }, + [TODO_MERGE] = { 'm', "merge" }, + [TODO_NOOP] = { 0, "noop" }, + [TODO_DROP] = { 'd', "drop" }, + [TODO_COMMENT] = { 0, NULL }, }; static const char *command_to_string(const enum todo_command command) -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v3 4/8] sequencer: define array with enum values 2022-06-28 13:25 ` [PATCH v3 4/8] sequencer: define array with enum values Derrick Stolee via GitGitGadget @ 2022-06-28 21:02 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-28 21:02 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Derrick Stolee <derrickstolee@github.com> > > The todo_command_info array defines which strings match with which > todo_command enum values. The array is defined in the same order as the > enum values, but if one changed without the other, then we would have > unexpected results. > > Make it easier to see changes to the enum and this array by using the > enum values as the indices of the array. OK. It is a bit of shame that we cannot define this array and enum at the same time in a single construct, but this is much safer than the status quo. Thanks. > > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > sequencer.c | 28 ++++++++++++++-------------- > 1 file changed, 14 insertions(+), 14 deletions(-) > > diff --git a/sequencer.c b/sequencer.c > index 1094e146b99..2ffee79619c 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -1685,20 +1685,20 @@ static struct { > char c; > const char *str; > } todo_command_info[] = { > - { 'p', "pick" }, > - { 0, "revert" }, > - { 'e', "edit" }, > - { 'r', "reword" }, > - { 'f', "fixup" }, > - { 's', "squash" }, > - { 'x', "exec" }, > - { 'b', "break" }, > - { 'l', "label" }, > - { 't', "reset" }, > - { 'm', "merge" }, > - { 0, "noop" }, > - { 'd', "drop" }, > - { 0, NULL } > + [TODO_PICK] = { 'p', "pick" }, > + [TODO_REVERT] = { 0, "revert" }, > + [TODO_EDIT] = { 'e', "edit" }, > + [TODO_REWORD] = { 'r', "reword" }, > + [TODO_FIXUP] = { 'f', "fixup" }, > + [TODO_SQUASH] = { 's', "squash" }, > + [TODO_EXEC] = { 'x', "exec" }, > + [TODO_BREAK] = { 'b', "break" }, > + [TODO_LABEL] = { 'l', "label" }, > + [TODO_RESET] = { 't', "reset" }, > + [TODO_MERGE] = { 'm', "merge" }, > + [TODO_NOOP] = { 0, "noop" }, > + [TODO_DROP] = { 'd', "drop" }, > + [TODO_COMMENT] = { 0, NULL }, > }; > > static const char *command_to_string(const enum todo_command command) ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v3 5/8] sequencer: add update-ref command 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget ` (3 preceding siblings ...) 2022-06-28 13:25 ` [PATCH v3 4/8] sequencer: define array with enum values Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 ` Derrick Stolee via GitGitGadget 2022-06-30 9:39 ` Phillip Wood 2022-06-28 13:25 ` [PATCH v3 6/8] rebase: add --update-refs option Derrick Stolee via GitGitGadget ` (5 subsequent siblings) 10 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> Add the boilerplate for an "update-ref" command in the sequencer. This connects to the current no-op do_update_ref() which will be filled in after more connections are created. The syntax in the todo list will be "update-ref <ref-name>" to signal that we should store the current commit as the value for updating <ref-name> at the end of the rebase. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- rebase-interactive.c | 3 +++ sequencer.c | 14 +++++++++++++- sequencer.h | 1 + 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/rebase-interactive.c b/rebase-interactive.c index 22394224faa..1ff07647af3 100644 --- a/rebase-interactive.c +++ b/rebase-interactive.c @@ -57,6 +57,9 @@ void append_todo_help(int command_count, " create a merge commit using the original merge commit's\n" " message (or the oneline, if no original merge commit was\n" " specified); use -c <commit> to reword the commit message\n" +"u, update-ref <ref> = track a placeholder for the <ref> to be updated\n" +" to this position in the new commits. The <ref> is\n" +" updated at the end of the rebase\n" "\n" "These lines can be re-ordered; they are executed from top to bottom.\n"); unsigned edit_todo = !(shortrevisions && shortonto); diff --git a/sequencer.c b/sequencer.c index 2ffee79619c..0b61835d441 100644 --- a/sequencer.c +++ b/sequencer.c @@ -1696,6 +1696,7 @@ static struct { [TODO_LABEL] = { 'l', "label" }, [TODO_RESET] = { 't', "reset" }, [TODO_MERGE] = { 'm', "merge" }, + [TODO_UPDATE_REF] = { 'u', "update-ref" }, [TODO_NOOP] = { 0, "noop" }, [TODO_DROP] = { 'd', "drop" }, [TODO_COMMENT] = { 0, NULL }, @@ -2457,7 +2458,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item, command_to_string(item->command)); if (item->command == TODO_EXEC || item->command == TODO_LABEL || - item->command == TODO_RESET) { + item->command == TODO_RESET || item->command == TODO_UPDATE_REF) { item->commit = NULL; item->arg_offset = bol - buf; item->arg_len = (int)(eol - bol); @@ -4056,6 +4057,11 @@ leave_merge: return ret; } +static int do_update_ref(struct repository *r, const char *ref_name) +{ + return 0; +} + static int is_final_fixup(struct todo_list *todo_list) { int i = todo_list->current; @@ -4431,6 +4437,12 @@ static int pick_commits(struct repository *r, return error_with_patch(r, item->commit, arg, item->arg_len, opts, res, 0); + } else if (item->command == TODO_UPDATE_REF) { + struct strbuf ref = STRBUF_INIT; + strbuf_add(&ref, arg, item->arg_len); + if ((res = do_update_ref(r, ref.buf))) + reschedule = 1; + strbuf_release(&ref); } else if (!is_noop(item->command)) return error(_("unknown command %d"), item->command); diff --git a/sequencer.h b/sequencer.h index 3ae541bb145..2cf5c1b9a38 100644 --- a/sequencer.h +++ b/sequencer.h @@ -95,6 +95,7 @@ enum todo_command { TODO_LABEL, TODO_RESET, TODO_MERGE, + TODO_UPDATE_REF, /* commands that do nothing but are counted for reporting progress */ TODO_NOOP, TODO_DROP, -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v3 5/8] sequencer: add update-ref command 2022-06-28 13:25 ` [PATCH v3 5/8] sequencer: add update-ref command Derrick Stolee via GitGitGadget @ 2022-06-30 9:39 ` Phillip Wood 0 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-06-30 9:39 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee On 28/06/2022 14:25, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > Add the boilerplate for an "update-ref" command in the sequencer. This > connects to the current no-op do_update_ref() which will be filled in > after more connections are created. > > The syntax in the todo list will be "update-ref <ref-name>" to signal > that we should store the current commit as the value for updating > <ref-name> at the end of the rebase. > > [...] > @@ -4431,6 +4437,12 @@ static int pick_commits(struct repository *r, > return error_with_patch(r, item->commit, > arg, item->arg_len, > opts, res, 0); > + } else if (item->command == TODO_UPDATE_REF) { > + struct strbuf ref = STRBUF_INIT; > + strbuf_add(&ref, arg, item->arg_len); It's a bit of a shame we have to faff here just to get a NULL terminated string. It does not matter but using xmemdupz() & free() would be one less line. > + if ((res = do_update_ref(r, ref.buf))) > + reschedule = 1; Rescheduling here is good Best Wishes Phillip ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v3 6/8] rebase: add --update-refs option 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget ` (4 preceding siblings ...) 2022-06-28 13:25 ` [PATCH v3 5/8] sequencer: add update-ref command Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 ` Derrick Stolee via GitGitGadget 2022-06-28 21:09 ` Junio C Hamano ` (2 more replies) 2022-06-28 13:25 ` [PATCH v3 7/8] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget ` (4 subsequent siblings) 10 siblings, 3 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> When working on a large feature, it can be helpful to break that feature into multiple smaller parts that become reviewed in sequence. During development or during review, a change to one part of the feature could affect multiple of these parts. An interactive rebase can help adjust the multi-part "story" of the branch. However, if there are branches tracking the different parts of the feature, then rebasing the entire list of commits can create commits not reachable from those "sub branches". It can take a manual step to update those branches. Add a new --update-refs option to 'git rebase -i' that adds 'update-ref <ref>' steps to the todo file whenever a commit that is being rebased is decorated with that <ref>. At the very end, the rebase process updates all of the listed refs to the values stored during the rebase operation. Be sure to iterate after any squashing or fixups are placed. Update the branch only after those squashes and fixups are complete. This allows a --fixup commit at the tip of the feature to apply correctly to the sub branch, even if it is fixing up the most-recent commit in that part. One potential problem here is that refs decorating commits that are already marked as "fixup!" or "squash!" will not be included in this list. Generally, the reordering of the "fixup!" and "squash!" is likely to change the relative order of these refs, so it is not recommended. The workflow here is intended to allow these kinds of commits at the tip of the rebased branch while the other sub branches come along for the ride without intervention. This change update the documentation and builtin to accept the --update-refs option as well as updating the todo file with the 'update-ref' commands. Tests are added to ensure that these todo commands are added in the correct locations. A future change will update the behavior to actually update the refs at the end of the rebase sequence. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- Documentation/git-rebase.txt | 8 +++ builtin/rebase.c | 5 ++ sequencer.c | 107 ++++++++++++++++++++++++++++++++++ sequencer.h | 1 + t/t2407-worktree-heads.sh | 17 ++++++ t/t3404-rebase-interactive.sh | 70 ++++++++++++++++++++++ 6 files changed, 208 insertions(+) diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index 262fb01aec0..e7611b4089c 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -609,6 +609,13 @@ provided. Otherwise an explicit `--no-reschedule-failed-exec` at the start would be overridden by the presence of `rebase.rescheduleFailedExec=true` configuration. +--update-refs:: +--no-update-refs:: + Automatically force-update any branches that point to commits that + are being rebased. Any branches that are checked out in a worktree + or point to a `squash! ...` or `fixup! ...` commit are not updated + in this way. + INCOMPATIBLE OPTIONS -------------------- @@ -632,6 +639,7 @@ are incompatible with the following options: * --empty= * --reapply-cherry-picks * --edit-todo + * --update-refs * --root when used in combination with --onto In addition, the following pairs of options are incompatible: diff --git a/builtin/rebase.c b/builtin/rebase.c index 7ab50cda2ad..56d82a52106 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -102,6 +102,7 @@ struct rebase_options { int reschedule_failed_exec; int reapply_cherry_picks; int fork_point; + int update_refs; }; #define REBASE_OPTIONS_INIT { \ @@ -298,6 +299,7 @@ static int do_interactive_rebase(struct rebase_options *opts, unsigned flags) ret = complete_action(the_repository, &replay, flags, shortrevisions, opts->onto_name, opts->onto, &opts->orig_head, &commands, opts->autosquash, + opts->update_refs, &todo_list); } @@ -1124,6 +1126,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) OPT_BOOL(0, "autosquash", &options.autosquash, N_("move commits that begin with " "squash!/fixup! under -i")), + OPT_BOOL(0, "update-refs", &options.update_refs, + N_("update local refs that point to commits " + "that are being rebased")), { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"), N_("GPG-sign commits"), PARSE_OPT_OPTARG, NULL, (intptr_t) "" }, diff --git a/sequencer.c b/sequencer.c index 0b61835d441..915d87a0336 100644 --- a/sequencer.c +++ b/sequencer.c @@ -35,6 +35,7 @@ #include "commit-reach.h" #include "rebase-interactive.h" #include "reset.h" +#include "branch.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -5615,10 +5616,113 @@ static int skip_unnecessary_picks(struct repository *r, return 0; } +struct todo_add_branch_context { + struct todo_item *items; + size_t items_nr; + size_t items_alloc; + struct strbuf *buf; + struct commit *commit; + struct string_list refs_to_oids; +}; + +static int add_decorations_to_list(const struct commit *commit, + struct todo_add_branch_context *ctx) +{ + const struct name_decoration *decoration = get_name_decoration(&commit->object); + + while (decoration) { + struct todo_item *item; + const char *path; + size_t base_offset = ctx->buf->len; + + ALLOC_GROW(ctx->items, + ctx->items_nr + 1, + ctx->items_alloc); + item = &ctx->items[ctx->items_nr]; + memset(item, 0, sizeof(*item)); + + /* If the branch is checked out, then leave a comment instead. */ + if ((path = branch_checked_out(decoration->name))) { + item->command = TODO_COMMENT; + strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", + decoration->name, path); + } else { + struct string_list_item *sti; + item->command = TODO_UPDATE_REF; + strbuf_addf(ctx->buf, "%s\n", decoration->name); + + sti = string_list_append(&ctx->refs_to_oids, + decoration->name); + sti->util = oiddup(the_hash_algo->null_oid); + } + + item->offset_in_buf = base_offset; + item->arg_offset = base_offset; + item->arg_len = ctx->buf->len - base_offset; + ctx->items_nr++; + + decoration = decoration->next; + } + + return 0; +} + +/* + * For each 'pick' command, find out if the commit has a decoration in + * refs/heads/. If so, then add a 'label for-update-refs/' command. + */ +static int todo_list_add_update_ref_commands(struct todo_list *todo_list) +{ + int i; + static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP; + struct decoration_filter decoration_filter = { + .include_ref_pattern = &decorate_refs_include, + .exclude_ref_pattern = &decorate_refs_exclude, + .exclude_ref_config_pattern = &decorate_refs_exclude_config, + }; + struct todo_add_branch_context ctx = { + .buf = &todo_list->buf, + .refs_to_oids = STRING_LIST_INIT_DUP, + }; + + ctx.items_alloc = 2 * todo_list->nr + 1; + ALLOC_ARRAY(ctx.items, ctx.items_alloc); + + string_list_append(&decorate_refs_include, "refs/heads/"); + load_ref_decorations(&decoration_filter, 0); + + for (i = 0; i < todo_list->nr; ) { + struct todo_item *item = &todo_list->items[i]; + + /* insert ith item into new list */ + ALLOC_GROW(ctx.items, + ctx.items_nr + 1, + ctx.items_alloc); + + ctx.items[ctx.items_nr++] = todo_list->items[i++]; + + if (item->commit) { + ctx.commit = item->commit; + add_decorations_to_list(item->commit, &ctx); + } + } + + string_list_clear(&ctx.refs_to_oids, 1); + free(todo_list->items); + todo_list->items = ctx.items; + todo_list->nr = ctx.items_nr; + todo_list->alloc = ctx.items_alloc; + + return 0; +} + int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags, const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, + unsigned update_refs, struct todo_list *todo_list) { char shortonto[GIT_MAX_HEXSZ + 1]; @@ -5637,6 +5741,9 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla item->arg_len = item->arg_offset = item->flags = item->offset_in_buf = 0; } + if (update_refs && todo_list_add_update_ref_commands(todo_list)) + return -1; + if (autosquash && todo_list_rearrange_squash(todo_list)) return -1; diff --git a/sequencer.h b/sequencer.h index 2cf5c1b9a38..e671d7e0743 100644 --- a/sequencer.h +++ b/sequencer.h @@ -167,6 +167,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, + unsigned update_refs, struct todo_list *todo_list); int todo_list_rearrange_squash(struct todo_list *todo_list); diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index f1e9e172a0c..203997d92c6 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -151,4 +151,21 @@ test_expect_success 'refuse to overwrite when in error states' ' done ' +. "$TEST_DIRECTORY"/lib-rebase.sh + +test_expect_success !SANITIZE_LEAK 'refuse to overwrite during rebase with --update-refs' ' + git commit --fixup HEAD~2 --allow-empty && + ( + set_cat_todo_editor && + test_must_fail git rebase -i --update-refs HEAD~3 >todo && + ! grep "update-refs" todo + ) && + git branch -f allow-update HEAD~2 && + ( + set_cat_todo_editor && + test_must_fail git rebase -i --update-refs HEAD~3 >todo && + grep "update-ref refs/heads/allow-update" todo + ) +' + test_done diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index f31afd4a547..3cd20733bc8 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1743,6 +1743,76 @@ test_expect_success 'ORIG_HEAD is updated correctly' ' test_cmp_rev ORIG_HEAD test-orig-head@{1} ' +test_expect_success '--update-refs adds label and update-ref commands' ' + git checkout -b update-refs no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git commit --allow-empty --fixup=third && + git branch -f shared-tip && + ( + set_cat_todo_editor && + + cat >expect <<-EOF && + pick $(git log -1 --format=%h J) J + update-ref refs/heads/second + update-ref refs/heads/first + pick $(git log -1 --format=%h K) K + pick $(git log -1 --format=%h L) L + fixup $(git log -1 --format=%h update-refs) fixup! L # empty + update-ref refs/heads/third + pick $(git log -1 --format=%h M) M + update-ref refs/heads/no-conflict-branch + update-ref refs/heads/shared-tip + EOF + + test_must_fail git rebase -i --autosquash --update-refs primary >todo && + test_cmp expect todo + ) +' + +test_expect_success '--update-refs adds commands with --rebase-merges' ' + git checkout -b update-refs-with-merge no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git merge -m merge branch2 && + git branch -f merge-branch && + git commit --fixup=third --allow-empty && + ( + set_cat_todo_editor && + + cat >expect <<-EOF && + label onto + reset onto + pick $(git log -1 --format=%h branch2~1) F + pick $(git log -1 --format=%h branch2) I + update-ref refs/heads/branch2 + label merge + reset onto + pick $(git log -1 --format=%h refs/heads/second) J + update-ref refs/heads/second + update-ref refs/heads/first + pick $(git log -1 --format=%h refs/heads/third~1) K + pick $(git log -1 --format=%h refs/heads/third) L + fixup $(git log -1 --format=%h update-refs-with-merge) fixup! L # empty + update-ref refs/heads/third + pick $(git log -1 --format=%h HEAD~2) M + update-ref refs/heads/no-conflict-branch + merge -C $(git log -1 --format=%h HEAD~1) merge # merge + update-ref refs/heads/merge-branch + EOF + + test_must_fail git rebase -i --autosquash \ + --rebase-merges=rebase-cousins \ + --update-refs primary >todo && + + test_cmp expect todo + ) +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v3 6/8] rebase: add --update-refs option 2022-06-28 13:25 ` [PATCH v3 6/8] rebase: add --update-refs option Derrick Stolee via GitGitGadget @ 2022-06-28 21:09 ` Junio C Hamano 2022-06-29 13:03 ` Derrick Stolee 2022-07-01 9:20 ` Phillip Wood 2022-07-01 21:20 ` Elijah Newren 2 siblings, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-06-28 21:09 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Derrick Stolee <derrickstolee@github.com> > > When working on a large feature, it can be helpful to break that feature > into multiple smaller parts that become reviewed in sequence. During > development or during review, a change to one part of the feature could > affect multiple of these parts. An interactive rebase can help adjust > the multi-part "story" of the branch. > > However, if there are branches tracking the different parts of the > feature, then rebasing the entire list of commits can create commits not > reachable from those "sub branches". It can take a manual step to update > those branches. > > Add a new --update-refs option to 'git rebase -i' that adds 'update-ref > <ref>' steps to the todo file whenever a commit that is being rebased is > decorated with that <ref>. At the very end, the rebase process updates > all of the listed refs to the values stored during the rebase operation. > > Be sure to iterate after any squashing or fixups are placed. Update the > branch only after those squashes and fixups are complete. This allows a > --fixup commit at the tip of the feature to apply correctly to the sub > branch, even if it is fixing up the most-recent commit in that part. > > One potential problem here is that refs decorating commits that are > already marked as "fixup!" or "squash!" will not be included in this > list. Generally, the reordering of the "fixup!" and "squash!" is likely > to change the relative order of these refs, so it is not recommended. > The workflow here is intended to allow these kinds of commits at the tip > of the rebased branch while the other sub branches come along for the > ride without intervention. > > This change update the documentation and builtin to accept the > --update-refs option as well as updating the todo file with the > 'update-ref' commands. Tests are added to ensure that these todo > commands are added in the correct locations. OK, so command line option is `--update-refs` because the range being rebased could contain more than one branch tips, and the option causes one `update-ref` insn per such ref left in the todo list? I had to read the above twice to remind me what was going on. The design looks OK here but has "rebase-merge/update-refs" already came into the picture, or will it be in future steps? I lost track. Let's read on. Thanks. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 6/8] rebase: add --update-refs option 2022-06-28 21:09 ` Junio C Hamano @ 2022-06-29 13:03 ` Derrick Stolee 0 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee @ 2022-06-29 13:03 UTC (permalink / raw) To: Junio C Hamano, Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 6/28/22 5:09 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: ... >> This change update the documentation and builtin to accept the >> --update-refs option as well as updating the todo file with the >> 'update-ref' commands. Tests are added to ensure that these todo >> commands are added in the correct locations. > > OK, so command line option is `--update-refs` because the range > being rebased could contain more than one branch tips, and the > option causes one `update-ref` insn per such ref left in the todo > list? I had to read the above twice to remind me what was going on. > > The design looks OK here but has "rebase-merge/update-refs" already > came into the picture, or will it be in future steps? I lost track. Sorry, we have the ability to read the file (and tests that read an artificial version of the file for the branch_checked_out() logic) but have not written it from within Git yet. I can make it clearer what is _not_ in this patch and coming in a future change. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 6/8] rebase: add --update-refs option 2022-06-28 13:25 ` [PATCH v3 6/8] rebase: add --update-refs option Derrick Stolee via GitGitGadget 2022-06-28 21:09 ` Junio C Hamano @ 2022-07-01 9:20 ` Phillip Wood 2022-07-01 21:20 ` Elijah Newren 2 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-07-01 9:20 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee On 28/06/2022 14:25, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > When working on a large feature, it can be helpful to break that feature > into multiple smaller parts that become reviewed in sequence. During > development or during review, a change to one part of the feature could > affect multiple of these parts. An interactive rebase can help adjust > the multi-part "story" of the branch. > > However, if there are branches tracking the different parts of the > feature, then rebasing the entire list of commits can create commits not > reachable from those "sub branches". It can take a manual step to update > those branches. > > Add a new --update-refs option to 'git rebase -i' that adds 'update-ref > <ref>' steps to the todo file whenever a commit that is being rebased is > decorated with that <ref>. At the very end, the rebase process updates > all of the listed refs to the values stored during the rebase operation. > > Be sure to iterate after any squashing or fixups are placed. Update the > branch only after those squashes and fixups are complete. This allows a > --fixup commit at the tip of the feature to apply correctly to the sub > branch, even if it is fixing up the most-recent commit in that part. > > One potential problem here is that refs decorating commits that are > already marked as "fixup!" or "squash!" will not be included in this > list. Generally, the reordering of the "fixup!" and "squash!" is likely > to change the relative order of these refs, so it is not recommended. > The workflow here is intended to allow these kinds of commits at the tip > of the rebased branch while the other sub branches come along for the > ride without intervention. > > This change update the documentation and builtin to accept the > --update-refs option as well as updating the todo file with the > 'update-ref' commands. Tests are added to ensure that these todo > commands are added in the correct locations. > > A future change will update the behavior to actually update the refs > at the end of the rebase sequence. This looks quite good to me, my main concern is that we don't check what the user did when they edited the todo list which leaves a potentially long window when another process could update a branch that is being rebased. I've left some comments below, they're mostly me talking to myself while reading the code. > +struct todo_add_branch_context { > + struct todo_item *items; > + size_t items_nr; > + size_t items_alloc; > + struct strbuf *buf; > + struct commit *commit; > + struct string_list refs_to_oids; > +}; > + > +static int add_decorations_to_list(const struct commit *commit, > + struct todo_add_branch_context *ctx) > +{ > + const struct name_decoration *decoration = get_name_decoration(&commit->object); > + > + while (decoration) { > + struct todo_item *item; > + const char *path; > + size_t base_offset = ctx->buf->len; > + > + ALLOC_GROW(ctx->items, > + ctx->items_nr + 1, > + ctx->items_alloc); > + item = &ctx->items[ctx->items_nr]; > + memset(item, 0, sizeof(*item)); > + > + /* If the branch is checked out, then leave a comment instead. */ > + if ((path = branch_checked_out(decoration->name))) { > + item->command = TODO_COMMENT; > + strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", > + decoration->name, path); > + } else { > + struct string_list_item *sti; > + item->command = TODO_UPDATE_REF; > + strbuf_addf(ctx->buf, "%s\n", decoration->name); > + > + sti = string_list_append(&ctx->refs_to_oids, > + decoration->name); > + sti->util = oiddup(the_hash_algo->null_oid); We don't record where the branch is pointing, but we do stop checkout from checking out the branch. There is a small race window where another process can checkout the branch (or start rebasing a branch with 'git rebase <upstream> <branch>') after we've added it to this list but have not written it to disk. > + } > + > + item->offset_in_buf = base_offset; > + item->arg_offset = base_offset; > + item->arg_len = ctx->buf->len - base_offset; > + ctx->items_nr++; > + > + decoration = decoration->next; > + } > + > + return 0; > +} > + > +/* > + * For each 'pick' command, find out if the commit has a decoration in > + * refs/heads/. If so, then add a 'label for-update-refs/' command. > + */ > +static int todo_list_add_update_ref_commands(struct todo_list *todo_list) > +{ > + int i; > + static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; > + static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP; > + static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP; > + struct decoration_filter decoration_filter = { > + .include_ref_pattern = &decorate_refs_include, > + .exclude_ref_pattern = &decorate_refs_exclude, > + .exclude_ref_config_pattern = &decorate_refs_exclude_config, > + }; > + struct todo_add_branch_context ctx = { > + .buf = &todo_list->buf, > + .refs_to_oids = STRING_LIST_INIT_DUP, > + }; > + > + ctx.items_alloc = 2 * todo_list->nr + 1; We pre-allocate enough space for each commit to have an update-refs command added after it ... > + ALLOC_ARRAY(ctx.items, ctx.items_alloc); > + > + string_list_append(&decorate_refs_include, "refs/heads/"); > + load_ref_decorations(&decoration_filter, 0); > + > + for (i = 0; i < todo_list->nr; ) { > + struct todo_item *item = &todo_list->items[i]; > + > + /* insert ith item into new list */ > + ALLOC_GROW(ctx.items, > + ctx.items_nr + 1, > + ctx.items_alloc); But we are careful to grow the array if that space is not enough. > + ctx.items[ctx.items_nr++] = todo_list->items[i++]; > + > + if (item->commit) { > + ctx.commit = item->commit; > + add_decorations_to_list(item->commit, &ctx); > + } > + } > + > + string_list_clear(&ctx.refs_to_oids, 1); > + free(todo_list->items); > + todo_list->items = ctx.items; > + todo_list->nr = ctx.items_nr; > + todo_list->alloc = ctx.items_alloc; > + > + return 0; > +} > + > int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags, > const char *shortrevisions, const char *onto_name, > struct commit *onto, const struct object_id *orig_head, > struct string_list *commands, unsigned autosquash, > + unsigned update_refs, > struct todo_list *todo_list) > { > char shortonto[GIT_MAX_HEXSZ + 1]; > @@ -5637,6 +5741,9 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla > item->arg_len = item->arg_offset = item->flags = item->offset_in_buf = 0; > } > > + if (update_refs && todo_list_add_update_ref_commands(todo_list)) > + return -1; > + We add the update-ref commands before the user edits the todo list but do not check if they have added or removed any update-ref commands when they edit it. This means we may lock a branch unnecessarily if they deleted an update-ref command or fail to lock a branch in a timely manner if they add an update-ref command. It will eventually be locked when we process the update-ref command during the rebase but that could be quite a bit later if there are conflicts. If we were to add the update-ref commands before the user edited the list and then write the update-refs file after it has been edited that would avoid some of these problems but leaves a bigger race window when two process think they can safely update a branch. That could be addressed by checking the other worktrees again once we have written the file which we really need to do in order to check the user hasn't added an update-ref command for a branch that is already being rebased in another worktree. Best Wishes Phillip ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 6/8] rebase: add --update-refs option 2022-06-28 13:25 ` [PATCH v3 6/8] rebase: add --update-refs option Derrick Stolee via GitGitGadget 2022-06-28 21:09 ` Junio C Hamano 2022-07-01 9:20 ` Phillip Wood @ 2022-07-01 21:20 ` Elijah Newren 2022-07-04 12:56 ` Derrick Stolee 2 siblings, 1 reply; 144+ messages in thread From: Elijah Newren @ 2022-07-01 21:20 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood, Derrick Stolee On Tue, Jun 28, 2022 at 6:26 AM Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com> wrote: > > From: Derrick Stolee <derrickstolee@github.com> > > When working on a large feature, it can be helpful to break that feature > into multiple smaller parts that become reviewed in sequence. During > development or during review, a change to one part of the feature could > affect multiple of these parts. An interactive rebase can help adjust > the multi-part "story" of the branch. > > However, if there are branches tracking the different parts of the > feature, then rebasing the entire list of commits can create commits not > reachable from those "sub branches". It can take a manual step to update > those branches. > > Add a new --update-refs option to 'git rebase -i' that adds 'update-ref > <ref>' steps to the todo file whenever a commit that is being rebased is > decorated with that <ref>. At the very end, the rebase process updates > all of the listed refs to the values stored during the rebase operation. > > Be sure to iterate after any squashing or fixups are placed. Update the > branch only after those squashes and fixups are complete. This allows a > --fixup commit at the tip of the feature to apply correctly to the sub > branch, even if it is fixing up the most-recent commit in that part. > > One potential problem here is that refs decorating commits that are > already marked as "fixup!" or "squash!" will not be included in this > list. Generally, the reordering of the "fixup!" and "squash!" is likely > to change the relative order of these refs, so it is not recommended. > The workflow here is intended to allow these kinds of commits at the tip > of the rebased branch while the other sub branches come along for the > ride without intervention. > > This change update the documentation and builtin to accept the > --update-refs option as well as updating the todo file with the > 'update-ref' commands. Tests are added to ensure that these todo > commands are added in the correct locations. > > A future change will update the behavior to actually update the refs > at the end of the rebase sequence. > > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > Documentation/git-rebase.txt | 8 +++ > builtin/rebase.c | 5 ++ > sequencer.c | 107 ++++++++++++++++++++++++++++++++++ > sequencer.h | 1 + > t/t2407-worktree-heads.sh | 17 ++++++ > t/t3404-rebase-interactive.sh | 70 ++++++++++++++++++++++ > 6 files changed, 208 insertions(+) > > diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt > index 262fb01aec0..e7611b4089c 100644 > --- a/Documentation/git-rebase.txt > +++ b/Documentation/git-rebase.txt > @@ -609,6 +609,13 @@ provided. Otherwise an explicit `--no-reschedule-failed-exec` at the > start would be overridden by the presence of > `rebase.rescheduleFailedExec=true` configuration. > > +--update-refs:: > +--no-update-refs:: > + Automatically force-update any branches that point to commits that > + are being rebased. Any branches that are checked out in a worktree > + or point to a `squash! ...` or `fixup! ...` commit are not updated > + in this way. > + > INCOMPATIBLE OPTIONS > -------------------- > > @@ -632,6 +639,7 @@ are incompatible with the following options: > * --empty= > * --reapply-cherry-picks > * --edit-todo > + * --update-refs > * --root when used in combination with --onto > > In addition, the following pairs of options are incompatible: > diff --git a/builtin/rebase.c b/builtin/rebase.c > index 7ab50cda2ad..56d82a52106 100644 > --- a/builtin/rebase.c > +++ b/builtin/rebase.c > @@ -102,6 +102,7 @@ struct rebase_options { > int reschedule_failed_exec; > int reapply_cherry_picks; > int fork_point; > + int update_refs; > }; > > #define REBASE_OPTIONS_INIT { \ > @@ -298,6 +299,7 @@ static int do_interactive_rebase(struct rebase_options *opts, unsigned flags) > ret = complete_action(the_repository, &replay, flags, > shortrevisions, opts->onto_name, opts->onto, > &opts->orig_head, &commands, opts->autosquash, > + opts->update_refs, > &todo_list); > } > > @@ -1124,6 +1126,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) > OPT_BOOL(0, "autosquash", &options.autosquash, > N_("move commits that begin with " > "squash!/fixup! under -i")), > + OPT_BOOL(0, "update-refs", &options.update_refs, > + N_("update local refs that point to commits " > + "that are being rebased")), > { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"), > N_("GPG-sign commits"), > PARSE_OPT_OPTARG, NULL, (intptr_t) "" }, > diff --git a/sequencer.c b/sequencer.c > index 0b61835d441..915d87a0336 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -35,6 +35,7 @@ > #include "commit-reach.h" > #include "rebase-interactive.h" > #include "reset.h" > +#include "branch.h" > > #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" > > @@ -5615,10 +5616,113 @@ static int skip_unnecessary_picks(struct repository *r, > return 0; > } > > +struct todo_add_branch_context { > + struct todo_item *items; > + size_t items_nr; > + size_t items_alloc; > + struct strbuf *buf; > + struct commit *commit; > + struct string_list refs_to_oids; > +}; > + > +static int add_decorations_to_list(const struct commit *commit, > + struct todo_add_branch_context *ctx) > +{ > + const struct name_decoration *decoration = get_name_decoration(&commit->object); > + > + while (decoration) { > + struct todo_item *item; > + const char *path; > + size_t base_offset = ctx->buf->len; > + > + ALLOC_GROW(ctx->items, > + ctx->items_nr + 1, > + ctx->items_alloc); > + item = &ctx->items[ctx->items_nr]; > + memset(item, 0, sizeof(*item)); > + > + /* If the branch is checked out, then leave a comment instead. */ > + if ((path = branch_checked_out(decoration->name))) { > + item->command = TODO_COMMENT; > + strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", > + decoration->name, path); > + } else { > + struct string_list_item *sti; > + item->command = TODO_UPDATE_REF; > + strbuf_addf(ctx->buf, "%s\n", decoration->name); > + > + sti = string_list_append(&ctx->refs_to_oids, > + decoration->name); > + sti->util = oiddup(the_hash_algo->null_oid); > + } > + > + item->offset_in_buf = base_offset; > + item->arg_offset = base_offset; > + item->arg_len = ctx->buf->len - base_offset; > + ctx->items_nr++; > + > + decoration = decoration->next; > + } > + > + return 0; > +} > + > +/* > + * For each 'pick' command, find out if the commit has a decoration in Is this really limited to picks? If someone uses --autosquash and has a fixup or squash in the list, wouldn't this apply as well, or does all of this apply before the transformations to fixup/squash? Also, what if the user is doing --rebase-merges and there's a merge commit with a branch pointing at the merge. Would that be included? > + * refs/heads/. If so, then add a 'label for-update-refs/' command. > + */ > +static int todo_list_add_update_ref_commands(struct todo_list *todo_list) > +{ > + int i; > + static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; > + static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP; > + static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP; > + struct decoration_filter decoration_filter = { > + .include_ref_pattern = &decorate_refs_include, > + .exclude_ref_pattern = &decorate_refs_exclude, > + .exclude_ref_config_pattern = &decorate_refs_exclude_config, > + }; > + struct todo_add_branch_context ctx = { > + .buf = &todo_list->buf, > + .refs_to_oids = STRING_LIST_INIT_DUP, > + }; > + > + ctx.items_alloc = 2 * todo_list->nr + 1; > + ALLOC_ARRAY(ctx.items, ctx.items_alloc); > + > + string_list_append(&decorate_refs_include, "refs/heads/"); > + load_ref_decorations(&decoration_filter, 0); > + > + for (i = 0; i < todo_list->nr; ) { > + struct todo_item *item = &todo_list->items[i]; > + > + /* insert ith item into new list */ > + ALLOC_GROW(ctx.items, > + ctx.items_nr + 1, > + ctx.items_alloc); > + > + ctx.items[ctx.items_nr++] = todo_list->items[i++]; > + > + if (item->commit) { > + ctx.commit = item->commit; > + add_decorations_to_list(item->commit, &ctx); > + } > + } > + > + string_list_clear(&ctx.refs_to_oids, 1); > + free(todo_list->items); > + todo_list->items = ctx.items; > + todo_list->nr = ctx.items_nr; > + todo_list->alloc = ctx.items_alloc; > + > + return 0; > +} > + > int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags, > const char *shortrevisions, const char *onto_name, > struct commit *onto, const struct object_id *orig_head, > struct string_list *commands, unsigned autosquash, > + unsigned update_refs, > struct todo_list *todo_list) > { > char shortonto[GIT_MAX_HEXSZ + 1]; > @@ -5637,6 +5741,9 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla > item->arg_len = item->arg_offset = item->flags = item->offset_in_buf = 0; > } > > + if (update_refs && todo_list_add_update_ref_commands(todo_list)) > + return -1; > + > if (autosquash && todo_list_rearrange_squash(todo_list)) > return -1; > > diff --git a/sequencer.h b/sequencer.h > index 2cf5c1b9a38..e671d7e0743 100644 > --- a/sequencer.h > +++ b/sequencer.h > @@ -167,6 +167,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla > const char *shortrevisions, const char *onto_name, > struct commit *onto, const struct object_id *orig_head, > struct string_list *commands, unsigned autosquash, > + unsigned update_refs, > struct todo_list *todo_list); > int todo_list_rearrange_squash(struct todo_list *todo_list); > > diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh > index f1e9e172a0c..203997d92c6 100755 > --- a/t/t2407-worktree-heads.sh > +++ b/t/t2407-worktree-heads.sh > @@ -151,4 +151,21 @@ test_expect_success 'refuse to overwrite when in error states' ' > done > ' > > +. "$TEST_DIRECTORY"/lib-rebase.sh > + > +test_expect_success !SANITIZE_LEAK 'refuse to overwrite during rebase with --update-refs' ' > + git commit --fixup HEAD~2 --allow-empty && > + ( > + set_cat_todo_editor && > + test_must_fail git rebase -i --update-refs HEAD~3 >todo && > + ! grep "update-refs" todo > + ) && > + git branch -f allow-update HEAD~2 && > + ( > + set_cat_todo_editor && > + test_must_fail git rebase -i --update-refs HEAD~3 >todo && > + grep "update-ref refs/heads/allow-update" todo > + ) > +' > + > test_done > diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh > index f31afd4a547..3cd20733bc8 100755 > --- a/t/t3404-rebase-interactive.sh > +++ b/t/t3404-rebase-interactive.sh > @@ -1743,6 +1743,76 @@ test_expect_success 'ORIG_HEAD is updated correctly' ' > test_cmp_rev ORIG_HEAD test-orig-head@{1} > ' > > +test_expect_success '--update-refs adds label and update-ref commands' ' > + git checkout -b update-refs no-conflict-branch && > + git branch -f base HEAD~4 && > + git branch -f first HEAD~3 && > + git branch -f second HEAD~3 && > + git branch -f third HEAD~1 && > + git commit --allow-empty --fixup=third && > + git branch -f shared-tip && > + ( > + set_cat_todo_editor && > + > + cat >expect <<-EOF && > + pick $(git log -1 --format=%h J) J > + update-ref refs/heads/second > + update-ref refs/heads/first > + pick $(git log -1 --format=%h K) K > + pick $(git log -1 --format=%h L) L > + fixup $(git log -1 --format=%h update-refs) fixup! L # empty > + update-ref refs/heads/third > + pick $(git log -1 --format=%h M) M > + update-ref refs/heads/no-conflict-branch > + update-ref refs/heads/shared-tip > + EOF > + > + test_must_fail git rebase -i --autosquash --update-refs primary >todo && > + test_cmp expect todo > + ) > +' > + > +test_expect_success '--update-refs adds commands with --rebase-merges' ' > + git checkout -b update-refs-with-merge no-conflict-branch && > + git branch -f base HEAD~4 && > + git branch -f first HEAD~3 && > + git branch -f second HEAD~3 && > + git branch -f third HEAD~1 && > + git merge -m merge branch2 && > + git branch -f merge-branch && > + git commit --fixup=third --allow-empty && > + ( > + set_cat_todo_editor && > + > + cat >expect <<-EOF && > + label onto > + reset onto > + pick $(git log -1 --format=%h branch2~1) F > + pick $(git log -1 --format=%h branch2) I > + update-ref refs/heads/branch2 > + label merge > + reset onto > + pick $(git log -1 --format=%h refs/heads/second) J > + update-ref refs/heads/second > + update-ref refs/heads/first > + pick $(git log -1 --format=%h refs/heads/third~1) K > + pick $(git log -1 --format=%h refs/heads/third) L > + fixup $(git log -1 --format=%h update-refs-with-merge) fixup! L # empty > + update-ref refs/heads/third > + pick $(git log -1 --format=%h HEAD~2) M > + update-ref refs/heads/no-conflict-branch > + merge -C $(git log -1 --format=%h HEAD~1) merge # merge > + update-ref refs/heads/merge-branch > + EOF > + > + test_must_fail git rebase -i --autosquash \ > + --rebase-merges=rebase-cousins \ > + --update-refs primary >todo && > + > + test_cmp expect todo > + ) > +' > + > # This must be the last test in this file > test_expect_success '$EDITOR and friends are unchanged' ' > test_editor_unchanged > -- > gitgitgadget > ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 6/8] rebase: add --update-refs option 2022-07-01 21:20 ` Elijah Newren @ 2022-07-04 12:56 ` Derrick Stolee 2022-07-04 17:57 ` Elijah Newren 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-07-04 12:56 UTC (permalink / raw) To: Elijah Newren, Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood On 7/1/22 3:20 PM, Elijah Newren wrote: >> +/* >> + * For each 'pick' command, find out if the commit has a decoration in > Is this really limited to picks? If someone uses --autosquash and has > a fixup or squash in the list, wouldn't this apply as well, or does > all of this apply before the transformations to fixup/squash? Also, > what if the user is doing --rebase-merges and there's a merge commit > with a branch pointing at the merge. Would that be included? > >> + * refs/heads/. If so, then add a 'label for-update-refs/' command. >> + */ This is limited to picks (for now) mostly for the reason that the fixup! and squash! commits are probably getting reordered, even if they exist at the tip of some refs. The workflow I am optimizing for is to create fixup! and squash! commits at the tip of a long multi-topic series, and then those get reordered into the list. In that case, the only ref that is pointing at one of those commits is also the HEAD branch, so that is not updated using this mechanism. This is a case where I'm very interested in alternatives here, and maybe I should clearly mark this option as experimental so we can work out cases like this based on use. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 6/8] rebase: add --update-refs option 2022-07-04 12:56 ` Derrick Stolee @ 2022-07-04 17:57 ` Elijah Newren 2022-07-05 22:22 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Elijah Newren @ 2022-07-04 17:57 UTC (permalink / raw) To: Derrick Stolee Cc: Derrick Stolee via GitGitGadget, Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood On Mon, Jul 4, 2022 at 5:56 AM Derrick Stolee <derrickstolee@github.com> wrote: > > On 7/1/22 3:20 PM, Elijah Newren wrote: > >> +/* > >> + * For each 'pick' command, find out if the commit has a decoration in > > Is this really limited to picks? If someone uses --autosquash and has > > a fixup or squash in the list, wouldn't this apply as well, or does > > all of this apply before the transformations to fixup/squash? Also, > > what if the user is doing --rebase-merges and there's a merge commit > > with a branch pointing at the merge. Would that be included? > > > >> + * refs/heads/. If so, then add a 'label for-update-refs/' command. > >> + */ > > This is limited to picks (for now) mostly for the reason that the > fixup! and squash! commits are probably getting reordered, even if > they exist at the tip of some refs. > > The workflow I am optimizing for is to create fixup! and squash! > commits at the tip of a long multi-topic series, and then those > get reordered into the list. In that case, the only ref that is > pointing at one of those commits is also the HEAD branch, so that > is not updated using this mechanism. > > This is a case where I'm very interested in alternatives here, and > maybe I should clearly mark this option as experimental so we can > work out cases like this based on use. So my question was that if I had the following `git log --oneline origin/main..` output: 555555 (HEAD -> branch2) fixup! second commit 444444 fourth commit 333333 third commit 222222 (branch1) second commit 111111 first commit and I run `git rebase -i --autosquash --update-refs origin/main`, what would I get? Is branch1 updated to the rebase of 222222, or does it include the amending from squashing 555555 into the rebased 222222? Does branch2 get "left behind" because it wasn't a pick commit at the tip of history, leading us to not update branch2 at all? Or does it get correctly pointed at the rebased version of 444444? Actually, I checked out ds/rebase-update-ref just now to try it, and it seems like it does the right thing: pick 111111 first commit pick 222222 second commit fixup 555555 fixup! second commit update-ref refs/heads/branch1 pick 333333 third commit pick 444444 fourth commit # Ref refs/heads/branch2 checked out at '...' The last line was very disorienting to me at first and made me think we had a bug, but the update-refs stuff is built on top of the normal rebase mechanism and branch2 will be updated by that logic rather than by the special update-refs handling. If I add another branch with a few commits on top of branch2, then branch2 is indeed updated and after the pick of 444444 (and the additional branch, say branch3, would be updated by the normal rebase logic instead of by the update-refs handling). So it all works correctly, but users might get worried or confused along the way wondering whether it will function correctly. Another part that users might find disorienting is that at the end, the rebase reports: Successfully rebased and updated refs/heads/branch2. which is correct but totally ignores the fact that it *also* rebased and updated other branches. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 6/8] rebase: add --update-refs option 2022-07-04 17:57 ` Elijah Newren @ 2022-07-05 22:22 ` Derrick Stolee 2022-07-08 2:27 ` Elijah Newren 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-07-05 22:22 UTC (permalink / raw) To: Elijah Newren Cc: Derrick Stolee via GitGitGadget, Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood On 7/4/22 11:57 AM, Elijah Newren wrote: > Actually, I checked out ds/rebase-update-ref just now to try it, and > it seems like it does the right thing: > > pick 111111 first commit > pick 222222 second commit > fixup 555555 fixup! second commit > update-ref refs/heads/branch1 > > pick 333333 third commit > pick 444444 fourth commit Thanks for trying it out! This is definitely the main goal of the feature, although it is also helpful when resolving conflicts or doing 'edit' steps. > # Ref refs/heads/branch2 checked out at '...' > > The last line was very disorienting to me at first and made me think > we had a bug, but the update-refs stuff is built on top of the normal > rebase mechanism and branch2 will be updated by that logic rather than > by the special update-refs handling. If I add another branch with a > few commits on top of branch2, then branch2 is indeed updated and > after the pick of 444444 (and the additional branch, say branch3, > would be updated by the normal rebase logic instead of by the > update-refs handling). So it all works correctly, but users might get > worried or confused along the way wondering whether it will function > correctly. I'll add a patch that removes the comment in the case of the HEAD ref. Thanks for the idea! > Another part that users might find disorienting is that at the end, > the rebase reports: > Successfully rebased and updated refs/heads/branch2. > which is correct but totally ignores the fact that it *also* rebased > and updated other branches. Good point. I can add an extra message at the end (as well as a warning for any refs that did not properly update at the end). Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 6/8] rebase: add --update-refs option 2022-07-05 22:22 ` Derrick Stolee @ 2022-07-08 2:27 ` Elijah Newren 0 siblings, 0 replies; 144+ messages in thread From: Elijah Newren @ 2022-07-08 2:27 UTC (permalink / raw) To: Derrick Stolee Cc: Derrick Stolee via GitGitGadget, Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood On Tue, Jul 5, 2022 at 3:22 PM Derrick Stolee <derrickstolee@github.com> wrote: > > On 7/4/22 11:57 AM, Elijah Newren wrote: > > Actually, I checked out ds/rebase-update-ref just now to try it, and > > it seems like it does the right thing: > > > > pick 111111 first commit > > pick 222222 second commit > > fixup 555555 fixup! second commit > > update-ref refs/heads/branch1 > > > > pick 333333 third commit > > pick 444444 fourth commit > > Thanks for trying it out! This is definitely the main goal of the > feature, although it is also helpful when resolving conflicts or > doing 'edit' steps. > > > # Ref refs/heads/branch2 checked out at '...' > > > > The last line was very disorienting to me at first and made me think > > we had a bug, but the update-refs stuff is built on top of the normal > > rebase mechanism and branch2 will be updated by that logic rather than > > by the special update-refs handling. If I add another branch with a > > few commits on top of branch2, then branch2 is indeed updated and > > after the pick of 444444 (and the additional branch, say branch3, > > would be updated by the normal rebase logic instead of by the > > update-refs handling). So it all works correctly, but users might get > > worried or confused along the way wondering whether it will function > > correctly. > > I'll add a patch that removes the comment in the case of the HEAD > ref. Thanks for the idea! Thanks, that'd improve things. I'm curious whether it'd be even better to have an update-ref line for HEAD, so that users don't wonder whether it's omitted from the updates. (That would leave an open question whether you filter out HEAD before actually calling update-ref, or use some other mechanism to make it all work behind the scenes.) > > Another part that users might find disorienting is that at the end, > > the rebase reports: > > Successfully rebased and updated refs/heads/branch2. > > which is correct but totally ignores the fact that it *also* rebased > > and updated other branches. > > Good point. I can add an extra message at the end (as well as a > warning for any refs that did not properly update at the end). Sounds good. ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v3 7/8] rebase: update refs from 'update-ref' commands 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget ` (5 preceding siblings ...) 2022-06-28 13:25 ` [PATCH v3 6/8] rebase: add --update-refs option Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 ` Derrick Stolee via GitGitGadget 2022-06-28 21:15 ` Junio C Hamano ` (3 more replies) 2022-06-28 13:25 ` [PATCH v3 8/8] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget ` (3 subsequent siblings) 10 siblings, 4 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The previous change introduced the 'git rebase --update-refs' option which added 'update-ref <ref>' commands to the todo list of an interactive rebase. Teach Git to record the HEAD position when reaching these 'update-ref' commands. The ref/OID pair is stored in the $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this file to avoid having other processes updating the refs in that file while the rebase is in progress. Not only do we update the file when the sequencer reaches these 'update-ref' commands, we then update the refs themselves at the end of the rebase sequence. If the rebase is aborted before this final step, then the refs are not updated. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 114 +++++++++++++++++++++++++++++++++- t/t3404-rebase-interactive.sh | 23 +++++++ 2 files changed, 136 insertions(+), 1 deletion(-) diff --git a/sequencer.c b/sequencer.c index 915d87a0336..4fd1c0b5bce 100644 --- a/sequencer.c +++ b/sequencer.c @@ -36,6 +36,7 @@ #include "rebase-interactive.h" #include "reset.h" #include "branch.h" +#include "log-tree.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -148,6 +149,14 @@ static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") */ static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") +/* + * The update-refs file stores a list of refs that will be updated at the end + * of the rebase sequence. The 'update-ref <ref>' commands in the todo file + * update the OIDs for the refs in this file, but the refs are not updated + * until the end of the rebase sequence. + */ +static GIT_PATH_FUNC(rebase_path_update_refs, "rebase-merge/update-refs") + /* * The following files are written by git-rebase just after parsing the * command-line. @@ -4058,11 +4067,110 @@ leave_merge: return ret; } -static int do_update_ref(struct repository *r, const char *ref_name) +static int write_update_refs_state(struct string_list *refs_to_oids) +{ + int result = 0; + FILE *fp = NULL; + struct string_list_item *item; + char *path = xstrdup(rebase_path_update_refs()); + + if (safe_create_leading_directories(path)) { + result = error(_("unable to create leading directories of %s"), + path); + goto cleanup; + } + + fp = fopen(path, "w"); + if (!fp) { + result = error_errno(_("could not open '%s' for writing"), path); + goto cleanup; + } + + for_each_string_list_item(item, refs_to_oids) + fprintf(fp, "%s\n%s\n", item->string, oid_to_hex(item->util)); + +cleanup: + if (fp) + fclose(fp); + return result; +} + +static int do_update_ref(struct repository *r, const char *refname) { + struct string_list_item *item; + struct string_list list = STRING_LIST_INIT_DUP; + int found = 0; + + sequencer_get_update_refs_state(r->gitdir, &list); + + for_each_string_list_item(item, &list) { + if (!strcmp(item->string, refname)) { + struct object_id oid; + free(item->util); + found = 1; + + if (!read_ref("HEAD", &oid)) { + item->util = oiddup(&oid); + break; + } + } + } + + if (!found) { + struct object_id oid; + item = string_list_append(&list, refname); + + if (!read_ref("HEAD", &oid)) + item->util = oiddup(&oid); + else + item->util = oiddup(the_hash_algo->null_oid); + } + + write_update_refs_state(&list); + string_list_clear(&list, 1); return 0; } +static int do_update_refs(struct repository *r) +{ + int res = 0; + struct string_list_item *item; + struct string_list refs_to_oids = STRING_LIST_INIT_DUP; + struct ref_store *refs = get_main_ref_store(r); + + sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); + + for_each_string_list_item(item, &refs_to_oids) { + struct object_id *oid_to = item->util; + struct object_id oid_from; + + if (oideq(oid_to, the_hash_algo->null_oid)) { + /* + * Ref was not updated. User may have deleted the + * 'update-ref' step. + */ + continue; + } + + if (read_ref(item->string, &oid_from)) { + /* + * The ref does not exist. The user probably + * inserted a new 'update-ref' step with a new + * branch name. + */ + oidcpy(&oid_from, the_hash_algo->null_oid); + } + + res |= refs_update_ref(refs, "rewritten during rebase", + item->string, + oid_to, &oid_from, + 0, UPDATE_REFS_MSG_ON_ERR); + } + + string_list_clear(&refs_to_oids, 1); + return res; +} + static int is_final_fixup(struct todo_list *todo_list) { int i = todo_list->current; @@ -4580,6 +4688,8 @@ cleanup_head_ref: strbuf_release(&head_ref); } + do_update_refs(r); + /* * Sequence of picks finished successfully; cleanup by * removing the .git/sequencer directory @@ -5709,6 +5819,8 @@ static int todo_list_add_update_ref_commands(struct todo_list *todo_list) } } + write_update_refs_state(&ctx.refs_to_oids); + string_list_clear(&ctx.refs_to_oids, 1); free(todo_list->items); todo_list->items = ctx.items; diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 3cd20733bc8..63a69bc073e 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1813,6 +1813,29 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' ) ' +compare_two_refs () { + git rev-parse $1 >expect && + git rev-parse $2 >actual && + test_cmp expect actual +} + +test_expect_success '--update-refs updates refs correctly' ' + git checkout -B update-refs no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + test_commit extra2 fileX && + git commit --amend --fixup=L && + + git rebase -i --autosquash --update-refs primary && + + compare_two_refs HEAD~3 refs/heads/first && + compare_two_refs HEAD~3 refs/heads/second && + compare_two_refs HEAD~1 refs/heads/third && + compare_two_refs HEAD refs/heads/no-conflict-branch +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v3 7/8] rebase: update refs from 'update-ref' commands 2022-06-28 13:25 ` [PATCH v3 7/8] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget @ 2022-06-28 21:15 ` Junio C Hamano 2022-06-29 13:05 ` Derrick Stolee 2022-06-29 13:06 ` Derrick Stolee ` (2 subsequent siblings) 3 siblings, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-06-28 21:15 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Derrick Stolee <derrickstolee@github.com> > > The previous change introduced the 'git rebase --update-refs' option > which added 'update-ref <ref>' commands to the todo list of an > interactive rebase. > > Teach Git to record the HEAD position when reaching these 'update-ref' > commands. The ref/OID pair is stored in the > $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this > file to avoid having other processes updating the refs in that file > while the rebase is in progress. > > Not only do we update the file when the sequencer reaches these > 'update-ref' commands, we then update the refs themselves at the end of > the rebase sequence. If the rebase is aborted before this final step, > then the refs are not updated. So, in general, we would * first scan the range of commits being rebased * compute what should happen and write the "todo" thing * write also "update-refs" to lick the branches to repel others * execute the "todo" insns one by one, possibly giving control back And this order is important---update-refs file is written fairly early, and the branches potentially involved in the rebase are all protected during the time-consuming (due to possibility of manual conflict resolution) execution step. Makes sense. > diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh > index 3cd20733bc8..63a69bc073e 100755 > --- a/t/t3404-rebase-interactive.sh > +++ b/t/t3404-rebase-interactive.sh > @@ -1813,6 +1813,29 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' > ) > ' > > +compare_two_refs () { > + git rev-parse $1 >expect && > + git rev-parse $2 >actual && > + test_cmp expect actual > +} > + > +test_expect_success '--update-refs updates refs correctly' ' > + git checkout -B update-refs no-conflict-branch && > + git branch -f base HEAD~4 && > + git branch -f first HEAD~3 && > + git branch -f second HEAD~3 && > + git branch -f third HEAD~1 && > + test_commit extra2 fileX && > + git commit --amend --fixup=L && > + > + git rebase -i --autosquash --update-refs primary && > + > + compare_two_refs HEAD~3 refs/heads/first && > + compare_two_refs HEAD~3 refs/heads/second && > + compare_two_refs HEAD~1 refs/heads/third && > + compare_two_refs HEAD refs/heads/no-conflict-branch > +' > + > # This must be the last test in this file > test_expect_success '$EDITOR and friends are unchanged' ' > test_editor_unchanged It would be nice to also have a test that makes sure that other people will be prevented from checking out a branch whose tips may be updated at the end. Thanks. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 7/8] rebase: update refs from 'update-ref' commands 2022-06-28 21:15 ` Junio C Hamano @ 2022-06-29 13:05 ` Derrick Stolee 2022-06-30 17:09 ` Junio C Hamano 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-06-29 13:05 UTC (permalink / raw) To: Junio C Hamano, Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 6/28/22 5:15 PM, Junio C Hamano wrote: > "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: ... > It would be nice to also have a test that makes sure that other > people will be prevented from checking out a branch whose tips may > be updated at the end. Patch 3 adds the tests that 'git branch -f' cannot update these refs, but I could add 'git checkout' tests, too. They both run the same branch_checked_out() helper, but they are doing different things once they have "access" to the branch. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 7/8] rebase: update refs from 'update-ref' commands 2022-06-29 13:05 ` Derrick Stolee @ 2022-06-30 17:09 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-30 17:09 UTC (permalink / raw) To: Derrick Stolee Cc: Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren Derrick Stolee <derrickstolee@github.com> writes: > On 6/28/22 5:15 PM, Junio C Hamano wrote: >> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > ... >> It would be nice to also have a test that makes sure that other >> people will be prevented from checking out a branch whose tips may >> be updated at the end. > > Patch 3 adds the tests that 'git branch -f' cannot update these refs, Ah, I didn't notice. We are covered then. Thanks. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 7/8] rebase: update refs from 'update-ref' commands 2022-06-28 13:25 ` [PATCH v3 7/8] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget 2022-06-28 21:15 ` Junio C Hamano @ 2022-06-29 13:06 ` Derrick Stolee 2022-07-01 9:31 ` Phillip Wood 2022-07-01 23:18 ` Elijah Newren 3 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee @ 2022-06-29 13:06 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 6/28/22 9:25 AM, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > +static int write_update_refs_state(struct string_list *refs_to_oids) > +{ > + int result = 0; > + FILE *fp = NULL; > + struct string_list_item *item; > + char *path = xstrdup(rebase_path_update_refs()); > + > + if (safe_create_leading_directories(path)) { > + result = error(_("unable to create leading directories of %s"), > + path); > + goto cleanup; > + } > + > + fp = fopen(path, "w"); I realized (while trying to go to sleep, of course) that I need to use a lockfile here. Parallel processes might be reading this file while we are writing it. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 7/8] rebase: update refs from 'update-ref' commands 2022-06-28 13:25 ` [PATCH v3 7/8] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget 2022-06-28 21:15 ` Junio C Hamano 2022-06-29 13:06 ` Derrick Stolee @ 2022-07-01 9:31 ` Phillip Wood 2022-07-01 18:35 ` Junio C Hamano 2022-07-01 23:18 ` Elijah Newren 3 siblings, 1 reply; 144+ messages in thread From: Phillip Wood @ 2022-07-01 9:31 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee Hi Stolee On 28/06/2022 14:25, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > The previous change introduced the 'git rebase --update-refs' option > which added 'update-ref <ref>' commands to the todo list of an > interactive rebase. > > Teach Git to record the HEAD position when reaching these 'update-ref' > commands. The ref/OID pair is stored in the > $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this > file to avoid having other processes updating the refs in that file > while the rebase is in progress. > > Not only do we update the file when the sequencer reaches these > 'update-ref' commands, we then update the refs themselves at the end of > the rebase sequence. If the rebase is aborted before this final step, > then the refs are not updated. This looks good, I've left a few comments but it seems basically sound to me. > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > sequencer.c | 114 +++++++++++++++++++++++++++++++++- > t/t3404-rebase-interactive.sh | 23 +++++++ > 2 files changed, 136 insertions(+), 1 deletion(-) > > diff --git a/sequencer.c b/sequencer.c > index 915d87a0336..4fd1c0b5bce 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -36,6 +36,7 @@ > #include "rebase-interactive.h" > #include "reset.h" > #include "branch.h" > +#include "log-tree.h" > > #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" > > @@ -148,6 +149,14 @@ static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") > */ > static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") > > +/* > + * The update-refs file stores a list of refs that will be updated at the end > + * of the rebase sequence. The 'update-ref <ref>' commands in the todo file > + * update the OIDs for the refs in this file, but the refs are not updated > + * until the end of the rebase sequence. > + */ > +static GIT_PATH_FUNC(rebase_path_update_refs, "rebase-merge/update-refs") > + > /* > * The following files are written by git-rebase just after parsing the > * command-line. > @@ -4058,11 +4067,110 @@ leave_merge: > return ret; > } > > -static int do_update_ref(struct repository *r, const char *ref_name) > +static int write_update_refs_state(struct string_list *refs_to_oids) > +{ > + int result = 0; > + FILE *fp = NULL; > + struct string_list_item *item; > + char *path = xstrdup(rebase_path_update_refs()); This is leaked > + if (safe_create_leading_directories(path)) { > + result = error(_("unable to create leading directories of %s"), > + path); > + goto cleanup; > + } > + > + fp = fopen(path, "w"); > + if (!fp) { > + result = error_errno(_("could not open '%s' for writing"), path); > + goto cleanup; > + } Can we use fopen_or_warn() here? It ignores ENOENT and ENOTDIR but I don't think that should matter here. > + > + for_each_string_list_item(item, refs_to_oids) > + fprintf(fp, "%s\n%s\n", item->string, oid_to_hex(item->util)); > + > +cleanup: > + if (fp) > + fclose(fp); > + return result; > +} > +compare_two_refs () { > + git rev-parse $1 >expect && > + git rev-parse $2 >actual && > + test_cmp expect actual > +} This is just test_cmp_rev > +test_expect_success '--update-refs updates refs correctly' ' > + git checkout -B update-refs no-conflict-branch && > + git branch -f base HEAD~4 && > + git branch -f first HEAD~3 && > + git branch -f second HEAD~3 && > + git branch -f third HEAD~1 && > + test_commit extra2 fileX && > + git commit --amend --fixup=L && > + > + git rebase -i --autosquash --update-refs primary && > + > + compare_two_refs HEAD~3 refs/heads/first && > + compare_two_refs HEAD~3 refs/heads/second && > + compare_two_refs HEAD~1 refs/heads/third && > + compare_two_refs HEAD refs/heads/no-conflict-branch > +' Do we need a new test for this or can we just check the refs at the end of one of the tests added in the last patch? > # This must be the last test in this file > test_expect_success '$EDITOR and friends are unchanged' ' > test_editor_unchanged I forgot to say on the last patch but you could maybe add a test_editor_unchanged at the end of t2407 now that there are tests which call test_set_editor Thanks for working on this, it will be a really useful addition to rebase. Phillip ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 7/8] rebase: update refs from 'update-ref' commands 2022-07-01 9:31 ` Phillip Wood @ 2022-07-01 18:35 ` Junio C Hamano 0 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-07-01 18:35 UTC (permalink / raw) To: Phillip Wood Cc: Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee Phillip Wood <phillip.wood123@gmail.com> writes: >> +static int write_update_refs_state(struct string_list *refs_to_oids) >> +{ >> + int result = 0; >> + FILE *fp = NULL; >> + struct string_list_item *item; >> + char *path = xstrdup(rebase_path_update_refs()); > > This is leaked Good eyes. >> + if (safe_create_leading_directories(path)) { >> + result = error(_("unable to create leading directories of %s"), >> + path); >> + goto cleanup; >> + } >> + >> + fp = fopen(path, "w"); >> + if (!fp) { >> + result = error_errno(_("could not open '%s' for writing"), path); >> + goto cleanup; >> + } > > Can we use fopen_or_warn() here? It ignores ENOENT and ENOTDIR but I > don't think that should matter here. fopen("no-such-directory/file", "w") would fail with ENOENT, and fopen("README.md/file", "w") would fail with ENOTDIR, I would think, so "should not matter because we are writing" is not the reason, but because we know the path is in a subdirectory of ".git/" that we know should exist, the most likely reason for the fopen to fail is because (1) the repository is broken (we will get ENOENT, ENOTDIR, which we want to warn about but fopen_or_warn() ignores, as well as other errors such as EISDIR), (2) the repository is unwritable (we will get EACCES), or (3) we are running low on diskspace (ENOSPC). I think that the fopen_or_warn() helper was primarily invented to read an optional file (so we deliberately ignore a failure to open one due to ENOENT and ENOTDIR), and we should be careful of its use for any other purpose, i.e. write access for any purpose and read access for files that we know we should be able to. >> +compare_two_refs () { >> + git rev-parse $1 >expect && >> + git rev-parse $2 >actual && >> + test_cmp expect actual >> +} > > This is just test_cmp_rev I love to see reviewers who know the existing API and helpers very well (including the fopen-or-warn above). Very much appreciated. > Thanks for working on this, it will be a really useful addition to rebase. Ditto. Thanks. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 7/8] rebase: update refs from 'update-ref' commands 2022-06-28 13:25 ` [PATCH v3 7/8] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget ` (2 preceding siblings ...) 2022-07-01 9:31 ` Phillip Wood @ 2022-07-01 23:18 ` Elijah Newren 2022-07-04 13:05 ` Derrick Stolee 3 siblings, 1 reply; 144+ messages in thread From: Elijah Newren @ 2022-07-01 23:18 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood, Derrick Stolee On Tue, Jun 28, 2022 at 6:26 AM Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com> wrote: > > From: Derrick Stolee <derrickstolee@github.com> > > The previous change introduced the 'git rebase --update-refs' option > which added 'update-ref <ref>' commands to the todo list of an > interactive rebase. > > Teach Git to record the HEAD position when reaching these 'update-ref' > commands. The ref/OID pair is stored in the > $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this > file to avoid having other processes updating the refs in that file > while the rebase is in progress. > > Not only do we update the file when the sequencer reaches these > 'update-ref' commands, we then update the refs themselves at the end of > the rebase sequence. If the rebase is aborted before this final step, > then the refs are not updated. Why update with each update-ref command? Couldn't the values be stored in memory and only written when we hit a conflict? > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > sequencer.c | 114 +++++++++++++++++++++++++++++++++- > t/t3404-rebase-interactive.sh | 23 +++++++ > 2 files changed, 136 insertions(+), 1 deletion(-) > > diff --git a/sequencer.c b/sequencer.c > index 915d87a0336..4fd1c0b5bce 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -36,6 +36,7 @@ > #include "rebase-interactive.h" > #include "reset.h" > #include "branch.h" > +#include "log-tree.h" > > #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" > > @@ -148,6 +149,14 @@ static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") > */ > static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") > > +/* > + * The update-refs file stores a list of refs that will be updated at the end > + * of the rebase sequence. The 'update-ref <ref>' commands in the todo file > + * update the OIDs for the refs in this file, but the refs are not updated > + * until the end of the rebase sequence. > + */ > +static GIT_PATH_FUNC(rebase_path_update_refs, "rebase-merge/update-refs") > + > /* > * The following files are written by git-rebase just after parsing the > * command-line. > @@ -4058,11 +4067,110 @@ leave_merge: > return ret; > } > > -static int do_update_ref(struct repository *r, const char *ref_name) > +static int write_update_refs_state(struct string_list *refs_to_oids) > +{ > + int result = 0; > + FILE *fp = NULL; > + struct string_list_item *item; > + char *path = xstrdup(rebase_path_update_refs()); > + > + if (safe_create_leading_directories(path)) { > + result = error(_("unable to create leading directories of %s"), > + path); > + goto cleanup; > + } > + > + fp = fopen(path, "w"); > + if (!fp) { > + result = error_errno(_("could not open '%s' for writing"), path); > + goto cleanup; > + } > + > + for_each_string_list_item(item, refs_to_oids) > + fprintf(fp, "%s\n%s\n", item->string, oid_to_hex(item->util)); Here you are storing the ref and the new oid to move it to. Any reason you don't store the previous oid for the branch? In particular, if someone hits a conflict, needs to resolve, and comes back some time much later and these intermediate branches have since moved on, should we actually update those branches? (And, if we do, should we at least give a warning?) > +cleanup: > + if (fp) > + fclose(fp); > + return result; > +} > + > +static int do_update_ref(struct repository *r, const char *refname) > { > + struct string_list_item *item; > + struct string_list list = STRING_LIST_INIT_DUP; > + int found = 0; > + > + sequencer_get_update_refs_state(r->gitdir, &list); > + > + for_each_string_list_item(item, &list) { > + if (!strcmp(item->string, refname)) { > + struct object_id oid; > + free(item->util); > + found = 1; > + > + if (!read_ref("HEAD", &oid)) { > + item->util = oiddup(&oid); > + break; > + } > + } > + } > + > + if (!found) { > + struct object_id oid; > + item = string_list_append(&list, refname); > + > + if (!read_ref("HEAD", &oid)) > + item->util = oiddup(&oid); > + else > + item->util = oiddup(the_hash_algo->null_oid); Seems a little unfortunate to not use a cached value from the latest commit we picked (and wrote to HEAD) and instead need to re-read HEAD. But, I guess sequencer is written to round-trip through files, so maybe optimizing this here doesn't make sense given all the other places in sequencer where we do lots of file reading and writing. > + } > + > + write_update_refs_state(&list); > + string_list_clear(&list, 1); > return 0; > } > > +static int do_update_refs(struct repository *r) > +{ > + int res = 0; > + struct string_list_item *item; > + struct string_list refs_to_oids = STRING_LIST_INIT_DUP; > + struct ref_store *refs = get_main_ref_store(r); > + > + sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); > + > + for_each_string_list_item(item, &refs_to_oids) { > + struct object_id *oid_to = item->util; > + struct object_id oid_from; > + > + if (oideq(oid_to, the_hash_algo->null_oid)) { > + /* > + * Ref was not updated. User may have deleted the > + * 'update-ref' step. > + */ > + continue; > + } > + > + if (read_ref(item->string, &oid_from)) { > + /* > + * The ref does not exist. The user probably > + * inserted a new 'update-ref' step with a new > + * branch name. > + */ > + oidcpy(&oid_from, the_hash_algo->null_oid); > + } > + > + res |= refs_update_ref(refs, "rewritten during rebase", > + item->string, > + oid_to, &oid_from, > + 0, UPDATE_REFS_MSG_ON_ERR); > + } > + > + string_list_clear(&refs_to_oids, 1); > + return res; > +} > + > static int is_final_fixup(struct todo_list *todo_list) > { > int i = todo_list->current; > @@ -4580,6 +4688,8 @@ cleanup_head_ref: > strbuf_release(&head_ref); > } > > + do_update_refs(r); > + > /* > * Sequence of picks finished successfully; cleanup by > * removing the .git/sequencer directory > @@ -5709,6 +5819,8 @@ static int todo_list_add_update_ref_commands(struct todo_list *todo_list) > } > } > > + write_update_refs_state(&ctx.refs_to_oids); > + > string_list_clear(&ctx.refs_to_oids, 1); > free(todo_list->items); > todo_list->items = ctx.items; > diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh > index 3cd20733bc8..63a69bc073e 100755 > --- a/t/t3404-rebase-interactive.sh > +++ b/t/t3404-rebase-interactive.sh > @@ -1813,6 +1813,29 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' > ) > ' > > +compare_two_refs () { > + git rev-parse $1 >expect && > + git rev-parse $2 >actual && > + test_cmp expect actual > +} > + > +test_expect_success '--update-refs updates refs correctly' ' > + git checkout -B update-refs no-conflict-branch && > + git branch -f base HEAD~4 && > + git branch -f first HEAD~3 && > + git branch -f second HEAD~3 && > + git branch -f third HEAD~1 && > + test_commit extra2 fileX && > + git commit --amend --fixup=L && > + > + git rebase -i --autosquash --update-refs primary && > + > + compare_two_refs HEAD~3 refs/heads/first && > + compare_two_refs HEAD~3 refs/heads/second && > + compare_two_refs HEAD~1 refs/heads/third && > + compare_two_refs HEAD refs/heads/no-conflict-branch > +' > + > # This must be the last test in this file > test_expect_success '$EDITOR and friends are unchanged' ' > test_editor_unchanged > -- > gitgitgadget > ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 7/8] rebase: update refs from 'update-ref' commands 2022-07-01 23:18 ` Elijah Newren @ 2022-07-04 13:05 ` Derrick Stolee 0 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee @ 2022-07-04 13:05 UTC (permalink / raw) To: Elijah Newren, Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood On 7/1/22 5:18 PM, Elijah Newren wrote: > On Tue, Jun 28, 2022 at 6:26 AM Derrick Stolee via GitGitGadget >> Not only do we update the file when the sequencer reaches these >> 'update-ref' commands, we then update the refs themselves at the end of >> the rebase sequence. If the rebase is aborted before this final step, >> then the refs are not updated. > > Why update with each update-ref command? Couldn't the values be > stored in memory and only written when we hit a conflict? I think that is a natural way to optimize the feature. I didn't look into that option as it seemed more error-prone to me. I'd be happy to be wrong here if an easy tweak makes this possible. >> -static int do_update_ref(struct repository *r, const char *ref_name) >> +static int write_update_refs_state(struct string_list *refs_to_oids) >> +{ >> + int result = 0; >> + FILE *fp = NULL; >> + struct string_list_item *item; >> + char *path = xstrdup(rebase_path_update_refs()); >> + >> + if (safe_create_leading_directories(path)) { >> + result = error(_("unable to create leading directories of %s"), >> + path); >> + goto cleanup; >> + } >> + >> + fp = fopen(path, "w"); >> + if (!fp) { >> + result = error_errno(_("could not open '%s' for writing"), path); >> + goto cleanup; >> + } >> + >> + for_each_string_list_item(item, refs_to_oids) >> + fprintf(fp, "%s\n%s\n", item->string, oid_to_hex(item->util)); > > Here you are storing the ref and the new oid to move it to. Any > reason you don't store the previous oid for the branch? In > particular, if someone hits a conflict, needs to resolve, and comes > back some time much later and these intermediate branches have since > moved on, should we actually update those branches? (And, if we do, > should we at least give a warning?) The branches don't move forward because other Git processes respect the update-refs file (this does not take into account third-party tools that don't know about that file, but we need to start somewhere). You're right that storing the old value would help us in this case where another tool updated the ref while we were in the middle of the rebase. The storage of the null OID is only to prevent writing over a ref when the user has removed the 'update-ref <ref>' command from the todo-list. It's probably better to remove the ref from the list when that happens. >> + if (!found) { >> + struct object_id oid; >> + item = string_list_append(&list, refname); >> + >> + if (!read_ref("HEAD", &oid)) >> + item->util = oiddup(&oid); >> + else >> + item->util = oiddup(the_hash_algo->null_oid); > > Seems a little unfortunate to not use a cached value from the latest > commit we picked (and wrote to HEAD) and instead need to re-read HEAD. > But, I guess sequencer is written to round-trip through files, so > maybe optimizing this here doesn't make sense given all the other > places in sequencer where we do lots of file reading and writing. It's also possible to optimize several things, but I tried to minimize the amount of change to the existing methods. This is my first foray into this part of the code, so I don't always know which information is readily available. I appreciate your attention here. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v3 8/8] rebase: add rebase.updateRefs config option 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget ` (6 preceding siblings ...) 2022-06-28 13:25 ` [PATCH v3 7/8] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 ` Derrick Stolee via GitGitGadget 2022-06-28 21:19 ` [PATCH v3 0/8] rebase: update branches in multi-part topic Junio C Hamano ` (2 subsequent siblings) 10 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-06-28 13:25 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The previous change added the --update-refs command-line option. For users who always want this mode, create the rebase.updateRefs config option which behaves the same way as rebase.autoSquash does with the --autosquash option. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- Documentation/config/rebase.txt | 3 +++ Documentation/git-rebase.txt | 3 +++ builtin/rebase.c | 5 +++++ t/t3404-rebase-interactive.sh | 14 ++++++++++++++ 4 files changed, 25 insertions(+) diff --git a/Documentation/config/rebase.txt b/Documentation/config/rebase.txt index 8c979cb20f2..f19bd0e0407 100644 --- a/Documentation/config/rebase.txt +++ b/Documentation/config/rebase.txt @@ -21,6 +21,9 @@ rebase.autoStash:: `--autostash` options of linkgit:git-rebase[1]. Defaults to false. +rebase.updateRefs:: + If set to true enable `--update-refs` option by default. + rebase.missingCommitsCheck:: If set to "warn", git rebase -i will print a warning if some commits are removed (e.g. a line was deleted), however the diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index e7611b4089c..1249f9ed617 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -615,6 +615,9 @@ start would be overridden by the presence of are being rebased. Any branches that are checked out in a worktree or point to a `squash! ...` or `fixup! ...` commit are not updated in this way. ++ +If the configuration variable `rebase.updateRefs` is set, then this option +can be used to override and disable this setting. INCOMPATIBLE OPTIONS -------------------- diff --git a/builtin/rebase.c b/builtin/rebase.c index 56d82a52106..8ebc98ea505 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -802,6 +802,11 @@ static int rebase_config(const char *var, const char *value, void *data) return 0; } + if (!strcmp(var, "rebase.updaterefs")) { + opts->update_refs = git_config_bool(var, value); + return 0; + } + if (!strcmp(var, "rebase.reschedulefailedexec")) { opts->reschedule_failed_exec = git_config_bool(var, value); return 0; diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 63a69bc073e..77478d94f3b 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1768,6 +1768,12 @@ test_expect_success '--update-refs adds label and update-ref commands' ' EOF test_must_fail git rebase -i --autosquash --update-refs primary >todo && + test_cmp expect todo && + + test_must_fail git -c rebase.autosquash=true \ + -c rebase.updaterefs=true \ + rebase -i primary >todo && + test_cmp expect todo ) ' @@ -1809,6 +1815,14 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' --rebase-merges=rebase-cousins \ --update-refs primary >todo && + test_cmp expect todo && + + test_must_fail git -c rebase.autosquash=true \ + -c rebase.updaterefs=true \ + rebase -i \ + --rebase-merges=rebase-cousins \ + primary >todo && + test_cmp expect todo ) ' -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v3 0/8] rebase: update branches in multi-part topic 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget ` (7 preceding siblings ...) 2022-06-28 13:25 ` [PATCH v3 8/8] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget @ 2022-06-28 21:19 ` Junio C Hamano 2022-07-01 13:43 ` Phillip Wood 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget 10 siblings, 0 replies; 144+ messages in thread From: Junio C Hamano @ 2022-06-28 21:19 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > This series adds a new --update-refs option to 'git rebase' (along with a > rebase.updateRefs config option) that adds 'update-ref' commands into the > TODO list. This is powered by the commit decoration machinery. I read through the patches and they made sense to me. Hopefully I can push out the integration result including this topic by the end of the day (today is actually my usual "every other Tuesday gitster goes offline" day, but I'll go offline tomorrow instead, so I want to injest as many reasonably-done topics as possible today). Thanks. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v3 0/8] rebase: update branches in multi-part topic 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget ` (8 preceding siblings ...) 2022-06-28 21:19 ` [PATCH v3 0/8] rebase: update branches in multi-part topic Junio C Hamano @ 2022-07-01 13:43 ` Phillip Wood 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget 10 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-07-01 13:43 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee Hi Stolee On 28/06/2022 14:25, Derrick Stolee via GitGitGadget wrote: > (Update: This is now based on ds/branch-checked-out.) > > This is a feature I've wanted for quite a while. When working on the sparse > index topic, I created a long RFC that actually broke into three topics for > full review upstream. These topics were sequential, so any feedback on an > earlier one required updates to the later ones. I would work on the full > feature and use interactive rebase to update the full list of commits. > However, I would need to update the branches pointing to those sub-topics. > > This series adds a new --update-refs option to 'git rebase' (along with a > rebase.updateRefs config option) that adds 'update-ref' commands into the > TODO list. This is powered by the commit decoration machinery. > > As an example, here is my in-progress bundle URI RFC split into subtopics as > they appear during the TODO list of a git rebase -i --update-refs: > > pick 2d966282ff3 docs: document bundle URI standard > pick 31396e9171a remote-curl: add 'get' capability > pick 54c6ab70f67 bundle-uri: create basic file-copy logic > pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// > pick 6adaf842684 fetch: add --bundle-uri option > pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration > update-ref refs/heads/bundle-redo/fetch I like the direction this is moving, my only concern with this version is how we handle the user adding and removing update-ref commands when the user edits the todo list. Best Wishes Phillip > pick 1e3f6546632 clone: add --bundle-uri option > pick 9e4a6fe9b68 clone: --bundle-uri cannot be combined with --depth > update-ref refs/heads/bundle-redo/clone > > pick 5451cb6599c bundle-uri: create bundle_list struct and helpers > pick 3029c3aca15 bundle-uri: create base key-value pair parsing > pick a8b2de79ce8 bundle-uri: create "key=value" line parsing > pick 92625a47673 bundle-uri: unit test "key=value" parsing > pick a8616af4dc2 bundle-uri: limit recursion depth for bundle lists > pick 9d6809a8d53 bundle-uri: parse bundle list in config format > pick 287a732b54c bundle-uri: fetch a list of bundles > update-ref refs/heads/bundle-redo/list > > pick b09f8226185 protocol v2: add server-side "bundle-uri" skeleton > pick 520204dcd1c bundle-uri client: add minimal NOOP client > pick 62e8b457b48 bundle-uri client: add "git ls-remote-bundle-uri" > pick 00eae925043 bundle-uri: serve URI advertisement from bundle.* config > pick 4277440a250 bundle-uri client: add boolean transfer.bundleURI setting > pick caf4599a81d bundle-uri: allow relative URLs in bundle lists > pick df255000b7e bundle-uri: download bundles from an advertised list > pick d71beabf199 clone: unbundle the advertised bundles > pick c9578391976 t5601: basic bundle URI tests > # Ref refs/heads/bundle-redo/rfc-3 checked out at '/home/stolee/_git/git-bundles' > > update-ref refs/heads/bundle-redo/advertise > > > Here is an outline of the series: > > * Patch 1 updates some tests for branch_checked_out() for the 'apply' > backend. > * Patch 2 updates branch_checked_out() to parse the > rebase-merge/update-refs file to block concurrent ref updates and > checkouts on branches selected by --update-refs. > * Patch 3 updates the todo list documentation to remove some unnecessary > dots in the 'merge' command. This makes it consistent with the 'fixup' > command before we document the 'update-ref' command. > * Patch 4 updates the definition of todo_command_info to use enum values as > array indices. > * Patches 5-7 implement the --update-refs logic itself. > * Patch 8 adds the rebase.updateRefs config option similar to > rebase.autoSquash. > > > Updates in v3 > ============= > > * The branch_checked_out() API was extracted to its own topic and is now > the ds/branch-checked-out branch. This series is now based on that one. > * The for_each_decoration() API was removed, since it became trivial once > it did not take a commit directly. > * The branch_checked_out() tests did not verify the rebase-apply data (for > the apply backend), so that is fixed. > * Instead of using the 'label' command and a final 'update-refs' command in > the todo list, use a new 'update-ref ' command. This command updates the > rebase-merge/update-refs file with the OID of HEAD at these steps. At the > very end of the rebase sequence, those refs are updated to the stored OID > values (assuming that they were not removed by the user, in which case we > notice that the OID is the null OID and we do nothing). > * New tests are added. > * The todo-list comment documentation has some new formatting updates, but > also includes a description of 'update-refs' in this version. > > > Updates in v2 > ============= > > As recommended by the excellent feedback, I have removed the 'exec' commands > in favor of the 'label' commands and a new 'update-refs' command at the very > end. This way, there is only one step that updates all of the refs at the > end instead of updating refs during the rebase. If a user runs 'git rebase > --abort' in the middle, then their refs are still where they need to be. > > Based on some of the discussion, it seemed like one way to do this would be > to have an 'update-ref ' command that would take the place of these 'label' > commands. However, this would require two things that make it a bit awkward: > > 1. We would need to replicate the storage of those positions during the > rebase. 'label' already does this pretty well. I've added the > "for-update-refs/" label to help here. > 2. If we want to close out all of the refs as the rebase is finishing, then > that "step" becomes invisible to the user (and a bit more complicated to > insert). Thus, the 'update-refs' step performs this action. If the user > wants to do things after that step, then they can do so by editing the > TODO list. > > Other updates: > > * The 'keep_decorations' parameter was renamed to 'update_refs'. > * I added tests for --rebase-merges=rebase-cousins to show how these labels > interact with other labels and merge commands. > * I changed the order of the insertion of these update-refs labels to be > before the fixups are rearranged. This fixes a bug where the tip commit > is a fixup! so its decorations are never inspected (and they would be in > the wrong place even if they were). The fixup! commands are properly > inserted between a pick and its following label command. Tests > demonstrate this is correct. > * Numerous style choices are updated based on feedback. > > Thank you for all of the detailed review and ideas in this space. I > appreciate any more ideas that can make this feature as effective as it can > be. > > Thanks, -Stolee > > Derrick Stolee (8): > t2407: test branches currently using apply backend > branch: consider refs under 'update-refs' > rebase-interactive: update 'merge' description > sequencer: define array with enum values > sequencer: add update-ref command > rebase: add --update-refs option > rebase: update refs from 'update-ref' commands > rebase: add rebase.updateRefs config option > > Documentation/config/rebase.txt | 3 + > Documentation/git-rebase.txt | 11 ++ > branch.c | 13 ++ > builtin/rebase.c | 10 ++ > rebase-interactive.c | 9 +- > sequencer.c | 303 ++++++++++++++++++++++++++++++-- > sequencer.h | 11 ++ > t/t2407-worktree-heads.sh | 52 +++++- > t/t3404-rebase-interactive.sh | 107 +++++++++++ > 9 files changed, 496 insertions(+), 23 deletions(-) > > > base-commit: 9bef0b1e6ec371e786c2fba3edcc06ad040a536c > Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1247%2Fderrickstolee%2Frebase-keep-decorations-v3 > Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1247/derrickstolee/rebase-keep-decorations-v3 > Pull-Request: https://github.com/gitgitgadget/git/pull/1247 > > Range-diff vs v2: > > 1: 4f9f3487641 < -: ----------- log-tree: create for_each_decoration() > 2: 5f54766e103 < -: ----------- branch: add branch_checked_out() helper > -: ----------- > 1: fbaedc7f1f0 t2407: test branches currently using apply backend > -: ----------- > 2: 2bc647b6fcd branch: consider refs under 'update-refs' > -: ----------- > 3: 669f4abd59e rebase-interactive: update 'merge' description > 3: 9f261c7df2c = 4: 6528a50343f sequencer: define array with enum values > 4: 842b2186d25 ! 5: e95ad41d355 sequencer: add update-refs command > @@ Metadata > Author: Derrick Stolee <derrickstolee@github.com> > > ## Commit message ## > - sequencer: add update-refs command > + sequencer: add update-ref command > > - Add the boilerplat for an "update-refs" command in the sequencer. This > - connects to the current no-op do_update_refs() which will be filled in > + Add the boilerplate for an "update-ref" command in the sequencer. This > + connects to the current no-op do_update_ref() which will be filled in > after more connections are created. > > + The syntax in the todo list will be "update-ref <ref-name>" to signal > + that we should store the current commit as the value for updating > + <ref-name> at the end of the rebase. > + > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > > + ## rebase-interactive.c ## > +@@ rebase-interactive.c: void append_todo_help(int command_count, > + " create a merge commit using the original merge commit's\n" > + " message (or the oneline, if no original merge commit was\n" > + " specified); use -c <commit> to reword the commit message\n" > ++"u, update-ref <ref> = track a placeholder for the <ref> to be updated\n" > ++" to this position in the new commits. The <ref> is\n" > ++" updated at the end of the rebase\n" > + "\n" > + "These lines can be re-ordered; they are executed from top to bottom.\n"); > + unsigned edit_todo = !(shortrevisions && shortonto); > + > ## sequencer.c ## > @@ sequencer.c: static struct { > [TODO_LABEL] = { 'l', "label" }, > [TODO_RESET] = { 't', "reset" }, > [TODO_MERGE] = { 'm', "merge" }, > -+ [TODO_UPDATE_REFS] = { 'u', "update-refs" }, > ++ [TODO_UPDATE_REF] = { 'u', "update-ref" }, > [TODO_NOOP] = { 0, "noop" }, > [TODO_DROP] = { 'd', "drop" }, > [TODO_COMMENT] = { 0, NULL }, > @@ sequencer.c: static int parse_insn_line(struct repository *r, struct todo_item *item, > - padding = strspn(bol, " \t"); > - bol += padding; > + command_to_string(item->command)); > > -- if (item->command == TODO_NOOP || item->command == TODO_BREAK) { > -+ if (item->command == TODO_NOOP || > -+ item->command == TODO_BREAK || > -+ item->command == TODO_UPDATE_REFS) { > - if (bol != eol) > - return error(_("%s does not accept arguments: '%s'"), > - command_to_string(item->command), bol); > + if (item->command == TODO_EXEC || item->command == TODO_LABEL || > +- item->command == TODO_RESET) { > ++ item->command == TODO_RESET || item->command == TODO_UPDATE_REF) { > + item->commit = NULL; > + item->arg_offset = bol - buf; > + item->arg_len = (int)(eol - bol); > @@ sequencer.c: leave_merge: > return ret; > } > > -+static int do_update_refs(struct repository *r) > ++static int do_update_ref(struct repository *r, const char *ref_name) > +{ > + return 0; > +} > @@ sequencer.c: static int pick_commits(struct repository *r, > return error_with_patch(r, item->commit, > arg, item->arg_len, > opts, res, 0); > -+ } else if (item->command == TODO_UPDATE_REFS) { > -+ if ((res = do_update_refs(r))) > ++ } else if (item->command == TODO_UPDATE_REF) { > ++ struct strbuf ref = STRBUF_INIT; > ++ strbuf_add(&ref, arg, item->arg_len); > ++ if ((res = do_update_ref(r, ref.buf))) > + reschedule = 1; > ++ strbuf_release(&ref); > } else if (!is_noop(item->command)) > return error(_("unknown command %d"), item->command); > > @@ sequencer.h: enum todo_command { > TODO_LABEL, > TODO_RESET, > TODO_MERGE, > -+ TODO_UPDATE_REFS, > ++ TODO_UPDATE_REF, > /* commands that do nothing but are counted for reporting progress */ > TODO_NOOP, > TODO_DROP, > 5: 0a4c110127b ! 6: 918b398d6a2 rebase: add --update-refs option > @@ Commit message > reachable from those "sub branches". It can take a manual step to update > those branches. > > - Add a new --update-refs option to 'git rebase -i' that adds 'label > - for-update-refs/*' steps to the todo file whenever a commit that is > - being rebased is decorated with that <ref>. At the very end, the > - 'update-refs' step is added to update all of the branches referenced by > - the 'label' steps. This allows the user to rebase a long list of commits > - in a multi-part feature and keep all of their pointers to those parts. > + Add a new --update-refs option to 'git rebase -i' that adds 'update-ref > + <ref>' steps to the todo file whenever a commit that is being rebased is > + decorated with that <ref>. At the very end, the rebase process updates > + all of the listed refs to the values stored during the rebase operation. > > - NOTE: This change only introduce the --update-refs option and implements > - the changes to the todo file. It does _not_ yet implement the action > - taken by the 'update-refs' todo step, which will be implemented and > - tested in a later change. > - > - Use the new for_each_decoration() while iterating over the existing todo > - list. Be sure to iterate after any squashing or fixups are placed. > - Update the branch only after those squashes and fixups are complete. > - This allows a --fixup commit at the tip of the feature to apply > - correctly to the sub branch, even if it is fixing up the most-recent > - commit in that part. > + Be sure to iterate after any squashing or fixups are placed. Update the > + branch only after those squashes and fixups are complete. This allows a > + --fixup commit at the tip of the feature to apply correctly to the sub > + branch, even if it is fixing up the most-recent commit in that part. > > One potential problem here is that refs decorating commits that are > already marked as "fixup!" or "squash!" will not be included in this > @@ Commit message > of the rebased branch while the other sub branches come along for the > ride without intervention. > > - Be careful to not attempt updating any branch that is checked out. The > - most common example is the branch being rebased is checked out and > - decorates the tip commit. If the user is rebasing commits reachable from > - a different branch that is checked out in a different worktree, then > - they may be surprised to not see that ref update. However, it's probably > - best to not optimize for this scenario and do the safest thing that will > - result in a successful rebase. A comment is left in the TODO list that > - signals that these refs are currently checked out. > + This change update the documentation and builtin to accept the > + --update-refs option as well as updating the todo file with the > + 'update-ref' commands. Tests are added to ensure that these todo > + commands are added in the correct locations. > + > + A future change will update the behavior to actually update the refs > + at the end of the rebase sequence. > > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > > @@ sequencer.c > #include "rebase-interactive.h" > #include "reset.h" > +#include "branch.h" > -+#include "log-tree.h" > > #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" > > @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, > + size_t items_alloc; > + struct strbuf *buf; > + struct commit *commit; > ++ struct string_list refs_to_oids; > +}; > + > -+static int add_branch_for_decoration(const struct name_decoration *d, void *data) > ++static int add_decorations_to_list(const struct commit *commit, > ++ struct todo_add_branch_context *ctx) > +{ > -+ struct todo_add_branch_context *ctx = data; > -+ size_t base_offset = ctx->buf->len; > -+ struct todo_item *item; > -+ char *path; > ++ const struct name_decoration *decoration = get_name_decoration(&commit->object); > + > -+ ALLOC_GROW(ctx->items, > -+ ctx->items_nr + 1, > -+ ctx->items_alloc); > -+ item = &ctx->items[ctx->items_nr]; > -+ memset(item, 0, sizeof(*item)); > ++ while (decoration) { > ++ struct todo_item *item; > ++ const char *path; > ++ size_t base_offset = ctx->buf->len; > + > -+ /* If the branch is checked out, then leave a comment instead. */ > -+ if (branch_checked_out(d->name, &path)) { > -+ item->command = TODO_COMMENT; > -+ strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", > -+ d->name, path); > -+ free(path); > -+ } else { > -+ item->command = TODO_LABEL; > -+ strbuf_addf(ctx->buf, "for-update-refs/%s\n", d->name); > -+ } > ++ ALLOC_GROW(ctx->items, > ++ ctx->items_nr + 1, > ++ ctx->items_alloc); > ++ item = &ctx->items[ctx->items_nr]; > ++ memset(item, 0, sizeof(*item)); > ++ > ++ /* If the branch is checked out, then leave a comment instead. */ > ++ if ((path = branch_checked_out(decoration->name))) { > ++ item->command = TODO_COMMENT; > ++ strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", > ++ decoration->name, path); > ++ } else { > ++ struct string_list_item *sti; > ++ item->command = TODO_UPDATE_REF; > ++ strbuf_addf(ctx->buf, "%s\n", decoration->name); > ++ > ++ sti = string_list_append(&ctx->refs_to_oids, > ++ decoration->name); > ++ sti->util = oiddup(the_hash_algo->null_oid); > ++ } > ++ > ++ item->offset_in_buf = base_offset; > ++ item->arg_offset = base_offset; > ++ item->arg_len = ctx->buf->len - base_offset; > ++ ctx->items_nr++; > + > -+ item->offset_in_buf = base_offset; > -+ item->arg_offset = base_offset; > -+ item->arg_len = ctx->buf->len - base_offset; > -+ ctx->items_nr++; > ++ decoration = decoration->next; > ++ } > + > + return 0; > +} > @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, > + struct decoration_filter decoration_filter = { > + .include_ref_pattern = &decorate_refs_include, > + .exclude_ref_pattern = &decorate_refs_exclude, > -+ .exclude_ref_config_pattern = &decorate_refs_exclude_config > ++ .exclude_ref_config_pattern = &decorate_refs_exclude_config, > + }; > + struct todo_add_branch_context ctx = { > + .buf = &todo_list->buf, > ++ .refs_to_oids = STRING_LIST_INIT_DUP, > + }; > + > + ctx.items_alloc = 2 * todo_list->nr + 1; > @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, > + > + if (item->commit) { > + ctx.commit = item->commit; > -+ for_each_decoration(item->commit, > -+ add_branch_for_decoration, > -+ &ctx); > ++ add_decorations_to_list(item->commit, &ctx); > + } > + } > + > -+ /* Add the "update-refs" step. */ > -+ ALLOC_GROW(ctx.items, > -+ ctx.items_nr + 1, > -+ ctx.items_alloc); > -+ memset(&ctx.items[ctx.items_nr], 0, sizeof(struct todo_item)); > -+ ctx.items[ctx.items_nr].command = TODO_UPDATE_REFS; > -+ ctx.items_nr++; > -+ > ++ string_list_clear(&ctx.refs_to_oids, 1); > + free(todo_list->items); > + todo_list->items = ctx.items; > + todo_list->nr = ctx.items_nr; > @@ sequencer.h: int complete_action(struct repository *r, struct replay_opts *opts, > int todo_list_rearrange_squash(struct todo_list *todo_list); > > > + ## t/t2407-worktree-heads.sh ## > +@@ t/t2407-worktree-heads.sh: test_expect_success 'refuse to overwrite when in error states' ' > + done > + ' > + > ++. "$TEST_DIRECTORY"/lib-rebase.sh > ++ > ++test_expect_success !SANITIZE_LEAK 'refuse to overwrite during rebase with --update-refs' ' > ++ git commit --fixup HEAD~2 --allow-empty && > ++ ( > ++ set_cat_todo_editor && > ++ test_must_fail git rebase -i --update-refs HEAD~3 >todo && > ++ ! grep "update-refs" todo > ++ ) && > ++ git branch -f allow-update HEAD~2 && > ++ ( > ++ set_cat_todo_editor && > ++ test_must_fail git rebase -i --update-refs HEAD~3 >todo && > ++ grep "update-ref refs/heads/allow-update" todo > ++ ) > ++' > ++ > + test_done > + > ## t/t3404-rebase-interactive.sh ## > @@ t/t3404-rebase-interactive.sh: test_expect_success 'ORIG_HEAD is updated correctly' ' > test_cmp_rev ORIG_HEAD test-orig-head@{1} > @@ t/t3404-rebase-interactive.sh: test_expect_success 'ORIG_HEAD is updated correct > + > + cat >expect <<-EOF && > + pick $(git log -1 --format=%h J) J > -+ label for-update-refs/refs/heads/second > -+ label for-update-refs/refs/heads/first > ++ update-ref refs/heads/second > ++ update-ref refs/heads/first > + pick $(git log -1 --format=%h K) K > + pick $(git log -1 --format=%h L) L > + fixup $(git log -1 --format=%h update-refs) fixup! L # empty > -+ label for-update-refs/refs/heads/third > ++ update-ref refs/heads/third > + pick $(git log -1 --format=%h M) M > -+ label for-update-refs/refs/heads/no-conflict-branch > -+ label for-update-refs/refs/heads/shared-tip > -+ update-refs > ++ update-ref refs/heads/no-conflict-branch > ++ update-ref refs/heads/shared-tip > + EOF > + > + test_must_fail git rebase -i --autosquash --update-refs primary >todo && > @@ t/t3404-rebase-interactive.sh: test_expect_success 'ORIG_HEAD is updated correct > + reset onto > + pick $(git log -1 --format=%h branch2~1) F > + pick $(git log -1 --format=%h branch2) I > -+ label for-update-refs/refs/heads/branch2 > ++ update-ref refs/heads/branch2 > + label merge > + reset onto > + pick $(git log -1 --format=%h refs/heads/second) J > -+ label for-update-refs/refs/heads/second > -+ label for-update-refs/refs/heads/first > ++ update-ref refs/heads/second > ++ update-ref refs/heads/first > + pick $(git log -1 --format=%h refs/heads/third~1) K > + pick $(git log -1 --format=%h refs/heads/third) L > + fixup $(git log -1 --format=%h update-refs-with-merge) fixup! L # empty > -+ label for-update-refs/refs/heads/third > ++ update-ref refs/heads/third > + pick $(git log -1 --format=%h HEAD~2) M > -+ label for-update-refs/refs/heads/no-conflict-branch > ++ update-ref refs/heads/no-conflict-branch > + merge -C $(git log -1 --format=%h HEAD~1) merge # merge > -+ label for-update-refs/refs/heads/merge-branch > -+ update-refs > ++ update-ref refs/heads/merge-branch > + EOF > + > + test_must_fail git rebase -i --autosquash \ > 6: 68f8e51b19c ! 7: 72e0481b643 sequencer: implement 'update-refs' command > @@ Metadata > Author: Derrick Stolee <derrickstolee@github.com> > > ## Commit message ## > - sequencer: implement 'update-refs' command > + rebase: update refs from 'update-ref' commands > > - The previous change allowed 'git rebase --update-refs' to create 'label' > - commands for each branch among the commits being rewritten and add an > - 'update-refs' command at the end of the todo list. Now, teach Git to > - update the refs during that final 'update-refs' command. > + The previous change introduced the 'git rebase --update-refs' option > + which added 'update-ref <ref>' commands to the todo list of an > + interactive rebase. > > - We need to create an array of new and old OIDs for each ref by iterating > - over the refs/rewritten/for-update-refs/ namespace. We cannot update the > - refs in-place since this will confuse the refs iterator. > + Teach Git to record the HEAD position when reaching these 'update-ref' > + commands. The ref/OID pair is stored in the > + $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this > + file to avoid having other processes updating the refs in that file > + while the rebase is in progress. > + > + Not only do we update the file when the sequencer reaches these > + 'update-ref' commands, we then update the refs themselves at the end of > + the rebase sequence. If the rebase is aborted before this final step, > + then the refs are not updated. > > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > > ## sequencer.c ## > +@@ > + #include "rebase-interactive.h" > + #include "reset.h" > + #include "branch.h" > ++#include "log-tree.h" > + > + #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" > + > +@@ sequencer.c: static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") > + */ > + static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") > + > ++/* > ++ * The update-refs file stores a list of refs that will be updated at the end > ++ * of the rebase sequence. The 'update-ref <ref>' commands in the todo file > ++ * update the OIDs for the refs in this file, but the refs are not updated > ++ * until the end of the rebase sequence. > ++ */ > ++static GIT_PATH_FUNC(rebase_path_update_refs, "rebase-merge/update-refs") > ++ > + /* > + * The following files are written by git-rebase just after parsing the > + * command-line. > @@ sequencer.c: leave_merge: > return ret; > } > > --static int do_update_refs(struct repository *r) > -+struct update_refs_context { > -+ struct ref_store *refs; > -+ char **ref_names; > -+ struct object_id *old; > -+ struct object_id *new; > -+ size_t nr; > -+ size_t alloc; > -+}; > -+ > -+static int add_ref_to_context(const char *refname, > -+ const struct object_id *oid, > -+ int flags, > -+ void *data) > +-static int do_update_ref(struct repository *r, const char *ref_name) > ++static int write_update_refs_state(struct string_list *refs_to_oids) > ++{ > ++ int result = 0; > ++ FILE *fp = NULL; > ++ struct string_list_item *item; > ++ char *path = xstrdup(rebase_path_update_refs()); > ++ > ++ if (safe_create_leading_directories(path)) { > ++ result = error(_("unable to create leading directories of %s"), > ++ path); > ++ goto cleanup; > ++ } > ++ > ++ fp = fopen(path, "w"); > ++ if (!fp) { > ++ result = error_errno(_("could not open '%s' for writing"), path); > ++ goto cleanup; > ++ } > ++ > ++ for_each_string_list_item(item, refs_to_oids) > ++ fprintf(fp, "%s\n%s\n", item->string, oid_to_hex(item->util)); > ++ > ++cleanup: > ++ if (fp) > ++ fclose(fp); > ++ return result; > ++} > ++ > ++static int do_update_ref(struct repository *r, const char *refname) > { > -+ int f = 0; > -+ const char *name; > -+ struct update_refs_context *ctx = data; > ++ struct string_list_item *item; > ++ struct string_list list = STRING_LIST_INIT_DUP; > ++ int found = 0; > ++ > ++ sequencer_get_update_refs_state(r->gitdir, &list); > + > -+ ALLOC_GROW(ctx->ref_names, ctx->nr + 1, ctx->alloc); > -+ ALLOC_GROW(ctx->old, ctx->nr + 1, ctx->alloc); > -+ ALLOC_GROW(ctx->new, ctx->nr + 1, ctx->alloc); > ++ for_each_string_list_item(item, &list) { > ++ if (!strcmp(item->string, refname)) { > ++ struct object_id oid; > ++ free(item->util); > ++ found = 1; > + > -+ if (!skip_prefix(refname, "refs/rewritten/for-update-refs/", &name)) > -+ return 1; > ++ if (!read_ref("HEAD", &oid)) { > ++ item->util = oiddup(&oid); > ++ break; > ++ } > ++ } > ++ } > + > -+ ctx->ref_names[ctx->nr] = xstrdup(name); > -+ oidcpy(&ctx->new[ctx->nr], oid); > -+ if (!refs_resolve_ref_unsafe(ctx->refs, name, 0, > -+ &ctx->old[ctx->nr], &f)) > -+ return 1; > ++ if (!found) { > ++ struct object_id oid; > ++ item = string_list_append(&list, refname); > + > -+ ctx->nr++; > ++ if (!read_ref("HEAD", &oid)) > ++ item->util = oiddup(&oid); > ++ else > ++ item->util = oiddup(the_hash_algo->null_oid); > ++ } > ++ > ++ write_update_refs_state(&list); > ++ string_list_clear(&list, 1); > return 0; > } > > +static int do_update_refs(struct repository *r) > +{ > -+ int i, res; > -+ struct update_refs_context ctx = { > -+ .refs = get_main_ref_store(r), > -+ .alloc = 16, > -+ }; > -+ ALLOC_ARRAY(ctx.ref_names, ctx.alloc); > -+ ALLOC_ARRAY(ctx.old, ctx.alloc); > -+ ALLOC_ARRAY(ctx.new, ctx.alloc); > -+ > -+ res = refs_for_each_fullref_in(ctx.refs, > -+ "refs/rewritten/for-update-refs/", > -+ add_ref_to_context, > -+ &ctx); > -+ > -+ for (i = 0; !res && i < ctx.nr; i++) > -+ res = refs_update_ref(ctx.refs, "rewritten during rebase", > -+ ctx.ref_names[i], > -+ &ctx.new[i], &ctx.old[i], > ++ int res = 0; > ++ struct string_list_item *item; > ++ struct string_list refs_to_oids = STRING_LIST_INIT_DUP; > ++ struct ref_store *refs = get_main_ref_store(r); > ++ > ++ sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); > ++ > ++ for_each_string_list_item(item, &refs_to_oids) { > ++ struct object_id *oid_to = item->util; > ++ struct object_id oid_from; > ++ > ++ if (oideq(oid_to, the_hash_algo->null_oid)) { > ++ /* > ++ * Ref was not updated. User may have deleted the > ++ * 'update-ref' step. > ++ */ > ++ continue; > ++ } > ++ > ++ if (read_ref(item->string, &oid_from)) { > ++ /* > ++ * The ref does not exist. The user probably > ++ * inserted a new 'update-ref' step with a new > ++ * branch name. > ++ */ > ++ oidcpy(&oid_from, the_hash_algo->null_oid); > ++ } > ++ > ++ res |= refs_update_ref(refs, "rewritten during rebase", > ++ item->string, > ++ oid_to, &oid_from, > + 0, UPDATE_REFS_MSG_ON_ERR); > ++ } > + > -+ for (i = 0; i < ctx.nr; i++) > -+ free(ctx.ref_names[i]); > -+ free(ctx.ref_names); > -+ free(ctx.old); > -+ free(ctx.new); > ++ string_list_clear(&refs_to_oids, 1); > + return res; > +} > + > static int is_final_fixup(struct todo_list *todo_list) > { > int i = todo_list->current; > +@@ sequencer.c: cleanup_head_ref: > + strbuf_release(&head_ref); > + } > + > ++ do_update_refs(r); > ++ > + /* > + * Sequence of picks finished successfully; cleanup by > + * removing the .git/sequencer directory > +@@ sequencer.c: static int todo_list_add_update_ref_commands(struct todo_list *todo_list) > + } > + } > + > ++ write_update_refs_state(&ctx.refs_to_oids); > ++ > + string_list_clear(&ctx.refs_to_oids, 1); > + free(todo_list->items); > + todo_list->items = ctx.items; > > ## t/t3404-rebase-interactive.sh ## > @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds commands with --rebase-merges' ' > @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds commands > + git branch -f third HEAD~1 && > + test_commit extra2 fileX && > + git commit --amend --fixup=L && > -+ ( > -+ git rebase -i --autosquash --update-refs primary && > -+ > -+ compare_two_refs HEAD~3 refs/heads/first && > -+ compare_two_refs HEAD~3 refs/heads/second && > -+ compare_two_refs HEAD~1 refs/heads/third && > -+ compare_two_refs HEAD refs/heads/no-conflict-branch > -+ ) > ++ > ++ git rebase -i --autosquash --update-refs primary && > ++ > ++ compare_two_refs HEAD~3 refs/heads/first && > ++ compare_two_refs HEAD~3 refs/heads/second && > ++ compare_two_refs HEAD~1 refs/heads/third && > ++ compare_two_refs HEAD refs/heads/no-conflict-branch > +' > + > # This must be the last test in this file > 7: 3d7d3f656b4 ! 8: d2cfdbfc431 rebase: add rebase.updateRefs config option > @@ Documentation/git-rebase.txt: start would be overridden by the presence of > or point to a `squash! ...` or `fixup! ...` commit are not updated > in this way. > ++ > -+If the `--update-refs` option is enabled by default using the > -+configuration variable `rebase.updateRefs`, this option can be > -+used to override and disable this setting. > ++If the configuration variable `rebase.updateRefs` is set, then this option > ++can be used to override and disable this setting. > > INCOMPATIBLE OPTIONS > -------------------- > ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v4 00/12] rebase: update branches in multi-part topic 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget ` (9 preceding siblings ...) 2022-07-01 13:43 ` Phillip Wood @ 2022-07-12 13:06 ` Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 01/12] t2407: test bisect and rebase as black-boxes Derrick Stolee via GitGitGadget ` (14 more replies) 10 siblings, 15 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee This series is based on ds/branch-checked-out. This is a feature I've wanted for quite a while. When working on the sparse index topic, I created a long RFC that actually broke into three topics for full review upstream. These topics were sequential, so any feedback on an earlier one required updates to the later ones. I would work on the full feature and use interactive rebase to update the full list of commits. However, I would need to update the branches pointing to those sub-topics. This series adds a new --update-refs option to 'git rebase' (along with a rebase.updateRefs config option) that adds 'update-ref' commands into the TODO list. This is powered by the commit decoration machinery. As an example, here is my in-progress bundle URI RFC split into subtopics as they appear during the TODO list of a git rebase -i --update-refs: pick 2d966282ff3 docs: document bundle URI standard pick 31396e9171a remote-curl: add 'get' capability pick 54c6ab70f67 bundle-uri: create basic file-copy logic pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// pick 6adaf842684 fetch: add --bundle-uri option pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration update-ref refs/heads/bundle-redo/fetch pick 1e3f6546632 clone: add --bundle-uri option pick 9e4a6fe9b68 clone: --bundle-uri cannot be combined with --depth update-ref refs/heads/bundle-redo/clone pick 5451cb6599c bundle-uri: create bundle_list struct and helpers pick 3029c3aca15 bundle-uri: create base key-value pair parsing pick a8b2de79ce8 bundle-uri: create "key=value" line parsing pick 92625a47673 bundle-uri: unit test "key=value" parsing pick a8616af4dc2 bundle-uri: limit recursion depth for bundle lists pick 9d6809a8d53 bundle-uri: parse bundle list in config format pick 287a732b54c bundle-uri: fetch a list of bundles update-ref refs/heads/bundle-redo/list pick b09f8226185 protocol v2: add server-side "bundle-uri" skeleton pick 520204dcd1c bundle-uri client: add minimal NOOP client pick 62e8b457b48 bundle-uri client: add "git ls-remote-bundle-uri" pick 00eae925043 bundle-uri: serve URI advertisement from bundle.* config pick 4277440a250 bundle-uri client: add boolean transfer.bundleURI setting pick caf4599a81d bundle-uri: allow relative URLs in bundle lists pick df255000b7e bundle-uri: download bundles from an advertised list pick d71beabf199 clone: unbundle the advertised bundles pick c9578391976 t5601: basic bundle URI tests # Ref refs/heads/bundle-redo/rfc-3 checked out at '/home/stolee/_git/git-bundles' update-ref refs/heads/bundle-redo/advertise Here is an outline of the series: * Patch 1 updates some tests for branch_checked_out() to use 'git bisect' and 'git rebase' as black-boxes instead of manually editing files inside $GIT_DIR. (Thanks, Junio!) * Patch 2 updates some tests for branch_checked_out() for the 'apply' backend. * Patch 3 updates branch_checked_out() to parse the rebase-merge/update-refs file to block concurrent ref updates and checkouts on branches selected by --update-refs. * Patch 4 updates the todo list documentation to remove some unnecessary dots in the 'merge' command. This makes it consistent with the 'fixup' command before we document the 'update-ref' command. * Patch 5 updates the definition of todo_command_info to use enum values as array indices. * Patches 6-8 implement the --update-refs logic itself. * Patch 9 specifically updates the update-refs file every time the user edits the todo-list (Thanks Phillip!) * Patch 10 adds the rebase.updateRefs config option similar to rebase.autoSquash. * Patch 11 ignores the HEAD ref when creating the todo list instead of making a comment (Thanks Elijah!) * Patch 12 adds messaging to the end of the rebase stating which refs were updated (Thanks Elijah!) Updates in v4 ============= This version took longer than I'd hoped (I had less time to work on it than anticipated) but it also has some major updates. These major updates are direct responses to the significant review this series has received. Thank you! * The update-refs file now stores "ref/before/after" triples (still separated by lines). This allows us to store the "before" OID of a ref in addition to the "after" that we will write to that ref at the end of the rebase. This allows us to do a "force-with-lease" update. The branch_checked_out() updates should prevent Git from updating those refs while under the rebase, but older versions and third-party tools don't have that protection. * The update-refs file is updated with every update to the todo-list file. This allows for some advanced changes to the file, including removing, adding, and duplicating 'update-ref' commands. * The message at the end of the rebase process now lists which refs were updated with the update-ref steps. This includes any ref updates that fail. * The branch_checked_out() tests now use 'git bisect' and 'git rebase' as black-boxes instead of testing their internals directly. Here are the more minor updates: * Dropped an unnecessary stat() call. * Updated commit messages to include extra details, based on confusion in last round. * The HEAD branch no longer appears as a comment line in the initial todo list. * The update-refs file is now written using a lockfile. * Tests now use test_cmp_rev. * A memory leak ('path' variable) is resolved. Updates in v3 ============= * The branch_checked_out() API was extracted to its own topic and is now the ds/branch-checked-out branch. This series is now based on that one. * The for_each_decoration() API was removed, since it became trivial once it did not take a commit directly. * The branch_checked_out() tests did not verify the rebase-apply data (for the apply backend), so that is fixed. * Instead of using the 'label' command and a final 'update-refs' command in the todo list, use a new 'update-ref ' command. This command updates the rebase-merge/update-refs file with the OID of HEAD at these steps. At the very end of the rebase sequence, those refs are updated to the stored OID values (assuming that they were not removed by the user, in which case we notice that the OID is the null OID and we do nothing). * New tests are added. * The todo-list comment documentation has some new formatting updates, but also includes a description of 'update-refs' in this version. Updates in v2 ============= As recommended by the excellent feedback, I have removed the 'exec' commands in favor of the 'label' commands and a new 'update-refs' command at the very end. This way, there is only one step that updates all of the refs at the end instead of updating refs during the rebase. If a user runs 'git rebase --abort' in the middle, then their refs are still where they need to be. Based on some of the discussion, it seemed like one way to do this would be to have an 'update-ref ' command that would take the place of these 'label' commands. However, this would require two things that make it a bit awkward: 1. We would need to replicate the storage of those positions during the rebase. 'label' already does this pretty well. I've added the "for-update-refs/" label to help here. 2. If we want to close out all of the refs as the rebase is finishing, then that "step" becomes invisible to the user (and a bit more complicated to insert). Thus, the 'update-refs' step performs this action. If the user wants to do things after that step, then they can do so by editing the TODO list. Other updates: * The 'keep_decorations' parameter was renamed to 'update_refs'. * I added tests for --rebase-merges=rebase-cousins to show how these labels interact with other labels and merge commands. * I changed the order of the insertion of these update-refs labels to be before the fixups are rearranged. This fixes a bug where the tip commit is a fixup! so its decorations are never inspected (and they would be in the wrong place even if they were). The fixup! commands are properly inserted between a pick and its following label command. Tests demonstrate this is correct. * Numerous style choices are updated based on feedback. Thank you for all of the detailed review and ideas in this space. I appreciate any more ideas that can make this feature as effective as it can be. Thanks, -Stolee Derrick Stolee (12): t2407: test bisect and rebase as black-boxes t2407: test branches currently using apply backend branch: consider refs under 'update-refs' rebase-interactive: update 'merge' description sequencer: define array with enum values sequencer: add update-ref command rebase: add --update-refs option rebase: update refs from 'update-ref' commands sequencer: rewrite update-refs as user edits todo list rebase: add rebase.updateRefs config option sequencer: ignore HEAD ref under --update-refs sequencer: notify user of --update-refs activity Documentation/config/rebase.txt | 3 + Documentation/git-rebase.txt | 11 + branch.c | 13 + builtin/rebase.c | 10 + rebase-interactive.c | 15 +- sequencer.c | 469 +++++++++++++++++++++++++++++++- sequencer.h | 23 ++ t/lib-rebase.sh | 15 + t/t2407-worktree-heads.sh | 103 +++++-- t/t3404-rebase-interactive.sh | 269 ++++++++++++++++++ 10 files changed, 887 insertions(+), 44 deletions(-) base-commit: 9bef0b1e6ec371e786c2fba3edcc06ad040a536c Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1247%2Fderrickstolee%2Frebase-keep-decorations-v4 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1247/derrickstolee/rebase-keep-decorations-v4 Pull-Request: https://github.com/gitgitgadget/git/pull/1247 Range-diff vs v3: -: ----------- > 1: 9e53a27017a t2407: test bisect and rebase as black-boxes 1: fbaedc7f1f0 ! 2: 540a3be256f t2407: test branches currently using apply backend @@ Commit message Signed-off-by: Derrick Stolee <derrickstolee@github.com> ## t/t2407-worktree-heads.sh ## -@@ t/t2407-worktree-heads.sh: test_expect_success 'refuse to overwrite: worktree in bisect' ' - grep "cannot force update the branch '\''fake-2'\'' checked out at.*wt-4" err +@@ t/t2407-worktree-heads.sh: test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in bisect' ' + grep "cannot force update the branch '\''wt-4'\'' checked out at.*wt-4" err ' --test_expect_success 'refuse to overwrite: worktree in rebase' ' -+test_expect_success 'refuse to overwrite: worktree in rebase (apply)' ' -+ test_when_finished rm -rf .git/worktrees/wt-*/rebase-apply && +-test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase' ' ++test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (apply)' ' ++ test_when_finished git -C wt-2 rebase --abort && + -+ mkdir -p .git/worktrees/wt-3/rebase-apply && -+ echo refs/heads/fake-1 >.git/worktrees/wt-3/rebase-apply/head-name && -+ echo refs/heads/fake-2 >.git/worktrees/wt-3/rebase-apply/onto && ++ # This will fail part-way through due to a conflict. ++ test_must_fail git -C wt-2 rebase --apply conflict-2 && + -+ test_must_fail git branch -f fake-1 HEAD 2>err && -+ grep "cannot force update the branch '\''fake-1'\'' checked out at.*wt-3" err ++ test_must_fail git branch -f wt-2 HEAD 2>err && ++ grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err +' + -+test_expect_success 'refuse to overwrite: worktree in rebase (merge)' ' - test_when_finished rm -rf .git/worktrees/wt-*/rebase-merge && ++test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (merge)' ' + test_when_finished git -C wt-2 rebase --abort && - mkdir -p .git/worktrees/wt-3/rebase-merge && + # This will fail part-way through due to a conflict. 2: 2bc647b6fcd ! 3: bf301a054e3 branch: consider refs under 'update-refs' @@ Commit message branch_checked_out(). The data store is a plaintext file inside the 'rebase-merge' directory - for that worktree. The file alternates refnames and OIDs. The OIDs will - be used to store the to-be-written values as the rebase progresses, but - can be ignored at the moment. + for that worktree. The file lists refnames followed by two OIDs, each on + separate lines. The OIDs will be used to store the original values of + the refs and the to-be-written values as the rebase progresses, but can + be ignored at the moment. Create a new sequencer_get_update_refs_state() method that parses this file and populates a struct string_list with the ref-OID pairs. We can @@ Commit message method. We can test that this works without having Git write this file by - artificially creating one in our test script. + artificially creating one in our test script, at least until 'git rebase + --update-refs' is implemented and we can use it directly. Signed-off-by: Derrick Stolee <derrickstolee@github.com> @@ branch.c: static void prepare_checked_out_branches(void) free_worktrees(worktrees); ## sequencer.c ## +@@ sequencer.c: static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") + */ + static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") + ++/* ++ * The update-refs file stores a list of refs that will be updated at the end ++ * of the rebase sequence. The 'update-ref <ref>' commands in the todo file ++ * update the OIDs for the refs in this file, but the refs are not updated ++ * until the end of the rebase sequence. ++ * ++ * rebase_path_update_refs() returns the path to this file for a given ++ * worktree directory. For the current worktree, pass the_repository->gitdir. ++ */ ++static char *rebase_path_update_refs(const char *wt_dir) ++{ ++ return xstrfmt("%s/rebase-merge/update-refs", wt_dir); ++} ++ + /* + * The following files are written by git-rebase just after parsing the + * command-line. +@@ sequencer.c: static GIT_PATH_FUNC(rebase_path_no_reschedule_failed_exec, "rebase-merge/no-res + static GIT_PATH_FUNC(rebase_path_drop_redundant_commits, "rebase-merge/drop_redundant_commits") + static GIT_PATH_FUNC(rebase_path_keep_redundant_commits, "rebase-merge/keep_redundant_commits") + ++/** ++ * A 'struct update_refs_record' represents a value in the update-refs ++ * list. We use a string_list to map refs to these (before, after) pairs. ++ */ ++struct update_ref_record { ++ struct object_id before; ++ struct object_id after; ++}; ++ + static int git_sequencer_config(const char *k, const char *v, void *cb) + { + struct replay_opts *opts = cb; @@ sequencer.c: int sequencer_determine_whence(struct repository *r, enum commit_whence *whence) return 0; @@ sequencer.c: int sequencer_determine_whence(struct repository *r, enum commit_wh + struct string_list *refs) +{ + int result = 0; -+ struct stat st; + FILE *fp = NULL; + struct strbuf ref = STRBUF_INIT; + struct strbuf hash = STRBUF_INIT; -+ char *path = xstrfmt("%s/rebase-merge/update-refs", wt_dir); ++ struct update_ref_record *rec = NULL; + -+ if (stat(path, &st)) -+ goto cleanup; ++ char *path = rebase_path_update_refs(wt_dir); + + fp = fopen(path, "r"); + if (!fp) + goto cleanup; + + while (strbuf_getline(&ref, fp) != EOF) { -+ struct object_id oid; + struct string_list_item *item; + ++ CALLOC_ARRAY(rec, 1); ++ + if (strbuf_getline(&hash, fp) == EOF || -+ get_oid_hex(hash.buf, &oid)) { ++ get_oid_hex(hash.buf, &rec->before)) { + warning(_("update-refs file at '%s' is invalid"), + path); + result = -1; + goto cleanup; + } + -+ item = string_list_append(refs, ref.buf); -+ item->util = oiddup(&oid); ++ if (strbuf_getline(&hash, fp) == EOF || ++ get_oid_hex(hash.buf, &rec->after)) { ++ warning(_("update-refs file at '%s' is invalid"), ++ path); ++ result = -1; ++ goto cleanup; ++ } ++ ++ item = string_list_insert(refs, ref.buf); ++ item->util = rec; ++ rec = NULL; + } + +cleanup: + if (fp) + fclose(fp); + free(path); ++ free(rec); + strbuf_release(&ref); + strbuf_release(&hash); + return result; @@ sequencer.h: void sequencer_post_commit_cleanup(struct repository *r, int verbos #endif /* SEQUENCER_H */ ## t/t2407-worktree-heads.sh ## -@@ t/t2407-worktree-heads.sh: TEST_PASSES_SANITIZE_LEAK=true +@@ t/t2407-worktree-heads.sh: test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer + grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err + ' - test_expect_success 'setup' ' - test_commit init && -- git branch -f fake-1 && -- git branch -f fake-2 && ++test_expect_success 'refuse to overwrite: worktree in rebase with --update-refs' ' ++ test_when_finished rm -rf .git/worktrees/wt-3/rebase-merge && ++ ++ mkdir -p .git/worktrees/wt-3/rebase-merge && ++ touch .git/worktrees/wt-3/rebase-merge/interactive && + -+ for i in 1 2 3 4 -+ do -+ git branch -f fake-$i || return 1 -+ done && - - for i in 1 2 3 4 - do -@@ t/t2407-worktree-heads.sh: test_expect_success 'refuse to overwrite: worktree in rebase (merge)' ' - echo refs/heads/fake-1 >.git/worktrees/wt-3/rebase-merge/head-name && - echo refs/heads/fake-2 >.git/worktrees/wt-3/rebase-merge/onto && - -- test_must_fail git branch -f fake-1 HEAD 2>err && -- grep "cannot force update the branch '\''fake-1'\'' checked out at.*wt-3" err + cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && + refs/heads/fake-3 + $(git rev-parse HEAD~1) ++ $(git rev-parse HEAD) + refs/heads/fake-4 + $(git rev-parse HEAD) ++ $(git rev-parse HEAD) + EOF + -+ for i in 1 3 4 ++ for i in 3 4 + do + test_must_fail git branch -f fake-$i HEAD 2>err && + grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || + return 1 + done - ' - ++' ++ test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: checked out' ' + test_must_fail git fetch server +refs/heads/wt-3:refs/heads/wt-3 2>err && + grep "refusing to fetch into branch '\''refs/heads/wt-3'\''" err && 3: 669f4abd59e ! 4: dec95681d2b rebase-interactive: update 'merge' description @@ Commit message The 'merge' command description for the todo list documentation in an interactive rebase has multiple lines. The lines other than the first one start with dots ('.') while the similar multi-line documentation for - 'fixup' does not. + 'fixup' does not. This description only appears in the comment text of + the todo file during an interactive rebase. The 'merge' command was documented when interactive rebase was first ported to C in 145e05ac44b (rebase -i: rewrite append_todo_help() in C, 4: 6528a50343f = 5: b2c09600918 sequencer: define array with enum values 5: e95ad41d355 = 6: fa7ecb718cf sequencer: add update-ref command 6: 918b398d6a2 ! 7: 3ec2cc922f9 rebase: add --update-refs option @@ Commit message 'update-ref' commands. Tests are added to ensure that these todo commands are added in the correct locations. - A future change will update the behavior to actually update the refs - at the end of the rebase sequence. + This change does _not_ include the actual behavior of tracking the + updated refs and writing the new ref values at the end of the rebase + process. That is deferred to a later change. Signed-off-by: Derrick Stolee <derrickstolee@github.com> @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, + item->command = TODO_UPDATE_REF; + strbuf_addf(ctx->buf, "%s\n", decoration->name); + -+ sti = string_list_append(&ctx->refs_to_oids, ++ sti = string_list_insert(&ctx->refs_to_oids, + decoration->name); + sti->util = oiddup(the_hash_algo->null_oid); + } @@ t/t2407-worktree-heads.sh: test_expect_success 'refuse to overwrite when in erro + grep "update-ref refs/heads/allow-update" todo + ) +' ++ ++# This must be the last test in this file ++test_expect_success '$EDITOR and friends are unchanged' ' ++ test_editor_unchanged ++' + test_done 7: 72e0481b643 ! 8: fb5f64c5201 rebase: update refs from 'update-ref' commands @@ Commit message interactive rebase. Teach Git to record the HEAD position when reaching these 'update-ref' - commands. The ref/OID pair is stored in the + commands. The ref/before/after triple is stored in the $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this file to avoid having other processes updating the refs in that file while the rebase is in progress. @@ Commit message Not only do we update the file when the sequencer reaches these 'update-ref' commands, we then update the refs themselves at the end of the rebase sequence. If the rebase is aborted before this final step, - then the refs are not updated. + then the refs are not updated. The 'before' value is used to ensure that + we do not accidentally obliterate a ref that was updated concurrently + (say, by an older version of Git or a third-party tool). + + Now that the 'git rebase --update-refs' command is implemented to write + to the update-refs file, we can remove the fake construction of the + update-refs file from a test in t2407-worktree-heads.sh. Signed-off-by: Derrick Stolee <derrickstolee@github.com> @@ sequencer.c #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" -@@ sequencer.c: static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") - */ - static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") +@@ sequencer.c: struct update_ref_record { + struct object_id after; + }; -+/* -+ * The update-refs file stores a list of refs that will be updated at the end -+ * of the rebase sequence. The 'update-ref <ref>' commands in the todo file -+ * update the OIDs for the refs in this file, but the refs are not updated -+ * until the end of the rebase sequence. -+ */ -+static GIT_PATH_FUNC(rebase_path_update_refs, "rebase-merge/update-refs") -+ - /* - * The following files are written by git-rebase just after parsing the - * command-line. ++static struct update_ref_record *init_update_ref_record(const char *ref) ++{ ++ struct update_ref_record *rec = xmalloc(sizeof(*rec)); ++ ++ oidcpy(&rec->before, null_oid()); ++ oidcpy(&rec->after, null_oid()); ++ ++ /* This may fail, but that's fine, we will keep the null OID. */ ++ read_ref(ref, &rec->before); ++ ++ return rec; ++} ++ + static int git_sequencer_config(const char *k, const char *v, void *cb) + { + struct replay_opts *opts = cb; @@ sequencer.c: leave_merge: return ret; } -static int do_update_ref(struct repository *r, const char *ref_name) +static int write_update_refs_state(struct string_list *refs_to_oids) -+{ + { + int result = 0; ++ struct lock_file lock = LOCK_INIT; + FILE *fp = NULL; + struct string_list_item *item; -+ char *path = xstrdup(rebase_path_update_refs()); ++ char *path; ++ ++ if (!refs_to_oids->nr) ++ return 0; ++ ++ path = rebase_path_update_refs(the_repository->gitdir); + + if (safe_create_leading_directories(path)) { + result = error(_("unable to create leading directories of %s"), @@ sequencer.c: leave_merge: + goto cleanup; + } + -+ fp = fopen(path, "w"); ++ if (hold_lock_file_for_update(&lock, path, 0) < 0) { ++ result = error(_("another 'rebase' process appears to be running; " ++ "'%s.lock' already exists"), ++ path); ++ goto cleanup; ++ } ++ ++ fp = fdopen_lock_file(&lock, "w"); + if (!fp) { + result = error_errno(_("could not open '%s' for writing"), path); ++ rollback_lock_file(&lock); + goto cleanup; + } + -+ for_each_string_list_item(item, refs_to_oids) -+ fprintf(fp, "%s\n%s\n", item->string, oid_to_hex(item->util)); ++ for_each_string_list_item(item, refs_to_oids) { ++ struct update_ref_record *rec = item->util; ++ fprintf(fp, "%s\n%s\n%s\n", item->string, ++ oid_to_hex(&rec->before), oid_to_hex(&rec->after)); ++ } ++ ++ result = commit_lock_file(&lock); + +cleanup: -+ if (fp) -+ fclose(fp); ++ free(path); + return result; +} + +static int do_update_ref(struct repository *r, const char *refname) - { ++{ + struct string_list_item *item; + struct string_list list = STRING_LIST_INIT_DUP; -+ int found = 0; + + sequencer_get_update_refs_state(r->gitdir, &list); + + for_each_string_list_item(item, &list) { + if (!strcmp(item->string, refname)) { -+ struct object_id oid; -+ free(item->util); -+ found = 1; -+ -+ if (!read_ref("HEAD", &oid)) { -+ item->util = oiddup(&oid); -+ break; -+ } ++ struct update_ref_record *rec = item->util; ++ read_ref("HEAD", &rec->after); ++ break; + } + } + -+ if (!found) { -+ struct object_id oid; -+ item = string_list_append(&list, refname); -+ -+ if (!read_ref("HEAD", &oid)) -+ item->util = oiddup(&oid); -+ else -+ item->util = oiddup(the_hash_algo->null_oid); -+ } -+ + write_update_refs_state(&list); + string_list_clear(&list, 1); return 0; @@ sequencer.c: leave_merge: + sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); + + for_each_string_list_item(item, &refs_to_oids) { -+ struct object_id *oid_to = item->util; -+ struct object_id oid_from; ++ struct update_ref_record *rec = item->util; + -+ if (oideq(oid_to, the_hash_algo->null_oid)) { ++ if (oideq(&rec->after, the_hash_algo->null_oid)) { + /* + * Ref was not updated. User may have deleted the + * 'update-ref' step. @@ sequencer.c: leave_merge: + continue; + } + -+ if (read_ref(item->string, &oid_from)) { -+ /* -+ * The ref does not exist. The user probably -+ * inserted a new 'update-ref' step with a new -+ * branch name. -+ */ -+ oidcpy(&oid_from, the_hash_algo->null_oid); -+ } -+ + res |= refs_update_ref(refs, "rewritten during rebase", -+ item->string, -+ oid_to, &oid_from, -+ 0, UPDATE_REFS_MSG_ON_ERR); ++ item->string, ++ &rec->after, &rec->before, ++ 0, UPDATE_REFS_MSG_ON_ERR); + } + + string_list_clear(&refs_to_oids, 1); @@ sequencer.c: cleanup_head_ref: /* * Sequence of picks finished successfully; cleanup by * removing the .git/sequencer directory +@@ sequencer.c: static int add_decorations_to_list(const struct commit *commit, + + sti = string_list_insert(&ctx->refs_to_oids, + decoration->name); +- sti->util = oiddup(the_hash_algo->null_oid); ++ sti->util = init_update_ref_record(decoration->name); + } + + item->offset_in_buf = base_offset; @@ sequencer.c: static int todo_list_add_update_ref_commands(struct todo_list *todo_list) } } @@ sequencer.c: static int todo_list_add_update_ref_commands(struct todo_list *todo free(todo_list->items); todo_list->items = ctx.items; + ## t/t2407-worktree-heads.sh ## +@@ t/t2407-worktree-heads.sh: test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer + grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err + ' + +-test_expect_success 'refuse to overwrite: worktree in rebase with --update-refs' ' +- test_when_finished rm -rf .git/worktrees/wt-3/rebase-merge && +- +- mkdir -p .git/worktrees/wt-3/rebase-merge && +- touch .git/worktrees/wt-3/rebase-merge/interactive && ++test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase with --update-refs' ' ++ test_when_finished git -C wt-3 rebase --abort && + +- cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && +- refs/heads/fake-3 +- $(git rev-parse HEAD~1) +- $(git rev-parse HEAD) +- refs/heads/fake-4 +- $(git rev-parse HEAD) +- $(git rev-parse HEAD) +- EOF ++ git branch -f can-be-updated wt-3 && ++ test_must_fail git -C wt-3 rebase --update-refs conflict-3 && + + for i in 3 4 + do +- test_must_fail git branch -f fake-$i HEAD 2>err && +- grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || ++ test_must_fail git branch -f can-be-updated HEAD 2>err && ++ grep "cannot force update the branch '\''can-be-updated'\'' checked out at.*wt-3" err || + return 1 + done + ' + ## t/t3404-rebase-interactive.sh ## @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds commands with --rebase-merges' ' ) ' -+compare_two_refs () { -+ git rev-parse $1 >expect && -+ git rev-parse $2 >actual && -+ test_cmp expect actual -+} -+ +test_expect_success '--update-refs updates refs correctly' ' + git checkout -B update-refs no-conflict-branch && + git branch -f base HEAD~4 && @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds commands + + git rebase -i --autosquash --update-refs primary && + -+ compare_two_refs HEAD~3 refs/heads/first && -+ compare_two_refs HEAD~3 refs/heads/second && -+ compare_two_refs HEAD~1 refs/heads/third && -+ compare_two_refs HEAD refs/heads/no-conflict-branch ++ test_cmp_rev HEAD~3 refs/heads/first && ++ test_cmp_rev HEAD~3 refs/heads/second && ++ test_cmp_rev HEAD~1 refs/heads/third && ++ test_cmp_rev HEAD refs/heads/no-conflict-branch +' + # This must be the last test in this file -: ----------- > 9: 29c7c76805a sequencer: rewrite update-refs as user edits todo list 8: d2cfdbfc431 = 10: c0022d07579 rebase: add rebase.updateRefs config option -: ----------- > 11: d53b4ff2cee sequencer: ignore HEAD ref under --update-refs -: ----------- > 12: d5cd4b49e46 sequencer: notify user of --update-refs activity -- gitgitgadget ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v4 01/12] t2407: test bisect and rebase as black-boxes 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 ` Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 02/12] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget ` (13 subsequent siblings) 14 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The tests added by d2ba271aad0 (branch: check for bisects and rebases, 2022-06-14) modified hidden state to verify the branch_checked_out() helper. While this indeed checks that the method implementation is _as designed_, it doesn't show that it is _correct_. Specifically, if 'git bisect' or 'git rebase' change their back-end for preserving refs, then these tests do not demonstrate that drift as a bug in branch_checked_out(). Modify the tests in t2407 to actually rely on a paused bisect or rebase. This requires adding the !SANITIZE_LEAK prereq for tests using those builtins. The logic is still tested for leaks in the final test which does set up that back-end directly for an error state that should not be possible using Git commands. Reported-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- t/t2407-worktree-heads.sh | 57 +++++++++++++++++++++------------------ 1 file changed, 31 insertions(+), 26 deletions(-) diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index b6be42f74a2..100ab286d5c 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -7,13 +7,18 @@ TEST_PASSES_SANITIZE_LEAK=true test_expect_success 'setup' ' test_commit init && - git branch -f fake-1 && - git branch -f fake-2 && for i in 1 2 3 4 do + git checkout -b conflict-$i && + echo "not I" >$i.t && + git add $i.t && + git commit -m "will conflict" && + + git checkout - && test_commit $i && git branch wt-$i && + git branch fake-$i && git worktree add wt-$i wt-$i || return 1 done && @@ -44,26 +49,26 @@ test_expect_success 'refuse to overwrite: checked out in worktree' ' done ' -test_expect_success 'refuse to overwrite: worktree in bisect' ' - test_when_finished rm -rf .git/worktrees/wt-*/BISECT_* && +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in bisect' ' + test_when_finished git -C wt-4 bisect reset && - touch .git/worktrees/wt-4/BISECT_LOG && - echo refs/heads/fake-2 >.git/worktrees/wt-4/BISECT_START && + # Set up a bisect so HEAD no longer points to wt-4. + git -C wt-4 bisect start && + git -C wt-4 bisect bad wt-4 && + git -C wt-4 bisect good wt-1 && - test_must_fail git branch -f fake-2 HEAD 2>err && - grep "cannot force update the branch '\''fake-2'\'' checked out at.*wt-4" err + test_must_fail git branch -f wt-4 HEAD 2>err && + grep "cannot force update the branch '\''wt-4'\'' checked out at.*wt-4" err ' -test_expect_success 'refuse to overwrite: worktree in rebase' ' - test_when_finished rm -rf .git/worktrees/wt-*/rebase-merge && +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase' ' + test_when_finished git -C wt-2 rebase --abort && - mkdir -p .git/worktrees/wt-3/rebase-merge && - touch .git/worktrees/wt-3/rebase-merge/interactive && - echo refs/heads/fake-1 >.git/worktrees/wt-3/rebase-merge/head-name && - echo refs/heads/fake-2 >.git/worktrees/wt-3/rebase-merge/onto && + # This will fail part-way through due to a conflict. + test_must_fail git -C wt-2 rebase conflict-2 && - test_must_fail git branch -f fake-1 HEAD 2>err && - grep "cannot force update the branch '\''fake-1'\'' checked out at.*wt-3" err + test_must_fail git branch -f wt-2 HEAD 2>err && + grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err ' test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: checked out' ' @@ -77,24 +82,24 @@ test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: checked out' ' ' test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: worktree in bisect' ' - test_when_finished rm -rf .git/worktrees/wt-*/BISECT_* && + test_when_finished git -C wt-4 bisect reset && - touch .git/worktrees/wt-4/BISECT_LOG && - echo refs/heads/fake-2 >.git/worktrees/wt-4/BISECT_START && + # Set up a bisect so HEAD no longer points to wt-4. + git -C wt-4 bisect start && + git -C wt-4 bisect bad wt-4 && + git -C wt-4 bisect good wt-1 && - test_must_fail git fetch server +refs/heads/fake-2:refs/heads/fake-2 2>err && + test_must_fail git fetch server +refs/heads/wt-4:refs/heads/wt-4 2>err && grep "refusing to fetch into branch" err ' test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: worktree in rebase' ' - test_when_finished rm -rf .git/worktrees/wt-*/rebase-merge && + test_when_finished git -C wt-3 rebase --abort && - mkdir -p .git/worktrees/wt-4/rebase-merge && - touch .git/worktrees/wt-4/rebase-merge/interactive && - echo refs/heads/fake-1 >.git/worktrees/wt-4/rebase-merge/head-name && - echo refs/heads/fake-2 >.git/worktrees/wt-4/rebase-merge/onto && + # This will fail part-way through due to a conflict. + test_must_fail git -C wt-3 rebase conflict-3 && - test_must_fail git fetch server +refs/heads/fake-1:refs/heads/fake-1 2>err && + test_must_fail git fetch server +refs/heads/wt-3:refs/heads/wt-3 2>err && grep "refusing to fetch into branch" err ' -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v4 02/12] t2407: test branches currently using apply backend 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 01/12] t2407: test bisect and rebase as black-boxes Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 ` Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 03/12] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget ` (12 subsequent siblings) 14 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The tests in t2407 that verify the branch_checked_out() helper in the case of bisects and rebases were added by 9347303db89 (branch: check for bisects and rebases, 2022-06-08). However, that commit failed to check for rebases that are using the 'apply' backend. Add a test that checks the apply backend. The implementation was already correct here, but it is good to have regression tests before modifying the implementation further. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- t/t2407-worktree-heads.sh | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index 100ab286d5c..a67ce5fb003 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -61,7 +61,17 @@ test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in bisect' ' grep "cannot force update the branch '\''wt-4'\'' checked out at.*wt-4" err ' -test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase' ' +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (apply)' ' + test_when_finished git -C wt-2 rebase --abort && + + # This will fail part-way through due to a conflict. + test_must_fail git -C wt-2 rebase --apply conflict-2 && + + test_must_fail git branch -f wt-2 HEAD 2>err && + grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err +' + +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (merge)' ' test_when_finished git -C wt-2 rebase --abort && # This will fail part-way through due to a conflict. -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v4 03/12] branch: consider refs under 'update-refs' 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 01/12] t2407: test bisect and rebase as black-boxes Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 02/12] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 ` Derrick Stolee via GitGitGadget 2022-07-15 15:37 ` Phillip Wood 2022-07-12 13:06 ` [PATCH v4 04/12] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget ` (11 subsequent siblings) 14 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The branch_checked_out() helper helps commands like 'git branch' and 'git fetch' from overwriting refs that are currently checked out in other worktrees. A future update to 'git rebase' will introduce a new '--update-refs' option which will update the local refs that point to commits that are being rebased. To avoid collisions as the rebase completes, we want to make the future data store for these refs to be considered by branch_checked_out(). The data store is a plaintext file inside the 'rebase-merge' directory for that worktree. The file lists refnames followed by two OIDs, each on separate lines. The OIDs will be used to store the original values of the refs and the to-be-written values as the rebase progresses, but can be ignored at the moment. Create a new sequencer_get_update_refs_state() method that parses this file and populates a struct string_list with the ref-OID pairs. We can then use this list to add to the current_checked_out_branches strmap used by branch_checked_out(). To properly navigate to the rebase directory for a given worktree, extract the static strbuf_worktree_gitdir() method to a public API method. We can test that this works without having Git write this file by artificially creating one in our test script, at least until 'git rebase --update-refs' is implemented and we can use it directly. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- branch.c | 13 +++++++ sequencer.c | 74 +++++++++++++++++++++++++++++++++++++++ sequencer.h | 9 +++++ t/t2407-worktree-heads.sh | 23 ++++++++++++ 4 files changed, 119 insertions(+) diff --git a/branch.c b/branch.c index 526e8237673..f252c4eef6c 100644 --- a/branch.c +++ b/branch.c @@ -365,6 +365,7 @@ static void prepare_checked_out_branches(void) char *old; struct wt_status_state state = { 0 }; struct worktree *wt = worktrees[i++]; + struct string_list update_refs = STRING_LIST_INIT_DUP; if (wt->is_bare) continue; @@ -400,6 +401,18 @@ static void prepare_checked_out_branches(void) strbuf_release(&ref); } wt_status_state_free_buffers(&state); + + if (!sequencer_get_update_refs_state(get_worktree_git_dir(wt), + &update_refs)) { + struct string_list_item *item; + for_each_string_list_item(item, &update_refs) { + old = strmap_put(¤t_checked_out_branches, + item->string, + xstrdup(wt->path)); + free(old); + } + string_list_clear(&update_refs, 1); + } } free_worktrees(worktrees); diff --git a/sequencer.c b/sequencer.c index 8c3ed3532ac..e93c61cbd25 100644 --- a/sequencer.c +++ b/sequencer.c @@ -147,6 +147,20 @@ static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") */ static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") +/* + * The update-refs file stores a list of refs that will be updated at the end + * of the rebase sequence. The 'update-ref <ref>' commands in the todo file + * update the OIDs for the refs in this file, but the refs are not updated + * until the end of the rebase sequence. + * + * rebase_path_update_refs() returns the path to this file for a given + * worktree directory. For the current worktree, pass the_repository->gitdir. + */ +static char *rebase_path_update_refs(const char *wt_dir) +{ + return xstrfmt("%s/rebase-merge/update-refs", wt_dir); +} + /* * The following files are written by git-rebase just after parsing the * command-line. @@ -169,6 +183,15 @@ static GIT_PATH_FUNC(rebase_path_no_reschedule_failed_exec, "rebase-merge/no-res static GIT_PATH_FUNC(rebase_path_drop_redundant_commits, "rebase-merge/drop_redundant_commits") static GIT_PATH_FUNC(rebase_path_keep_redundant_commits, "rebase-merge/keep_redundant_commits") +/** + * A 'struct update_refs_record' represents a value in the update-refs + * list. We use a string_list to map refs to these (before, after) pairs. + */ +struct update_ref_record { + struct object_id before; + struct object_id after; +}; + static int git_sequencer_config(const char *k, const char *v, void *cb) { struct replay_opts *opts = cb; @@ -5901,3 +5924,54 @@ int sequencer_determine_whence(struct repository *r, enum commit_whence *whence) return 0; } + +int sequencer_get_update_refs_state(const char *wt_dir, + struct string_list *refs) +{ + int result = 0; + FILE *fp = NULL; + struct strbuf ref = STRBUF_INIT; + struct strbuf hash = STRBUF_INIT; + struct update_ref_record *rec = NULL; + + char *path = rebase_path_update_refs(wt_dir); + + fp = fopen(path, "r"); + if (!fp) + goto cleanup; + + while (strbuf_getline(&ref, fp) != EOF) { + struct string_list_item *item; + + CALLOC_ARRAY(rec, 1); + + if (strbuf_getline(&hash, fp) == EOF || + get_oid_hex(hash.buf, &rec->before)) { + warning(_("update-refs file at '%s' is invalid"), + path); + result = -1; + goto cleanup; + } + + if (strbuf_getline(&hash, fp) == EOF || + get_oid_hex(hash.buf, &rec->after)) { + warning(_("update-refs file at '%s' is invalid"), + path); + result = -1; + goto cleanup; + } + + item = string_list_insert(refs, ref.buf); + item->util = rec; + rec = NULL; + } + +cleanup: + if (fp) + fclose(fp); + free(path); + free(rec); + strbuf_release(&ref); + strbuf_release(&hash); + return result; +} diff --git a/sequencer.h b/sequencer.h index da64473636b..3ae541bb145 100644 --- a/sequencer.h +++ b/sequencer.h @@ -232,4 +232,13 @@ void sequencer_post_commit_cleanup(struct repository *r, int verbose); int sequencer_get_last_command(struct repository* r, enum replay_action *action); int sequencer_determine_whence(struct repository *r, enum commit_whence *whence); + +/** + * Append the set of ref-OID pairs that are currently stored for the 'git + * rebase --update-refs' feature if such a rebase is currently happening. + * + * Localized to a worktree's git dir. + */ +int sequencer_get_update_refs_state(const char *wt_dir, struct string_list *refs); + #endif /* SEQUENCER_H */ diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index a67ce5fb003..97f5c87f8c8 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -81,6 +81,29 @@ test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err ' +test_expect_success 'refuse to overwrite: worktree in rebase with --update-refs' ' + test_when_finished rm -rf .git/worktrees/wt-3/rebase-merge && + + mkdir -p .git/worktrees/wt-3/rebase-merge && + touch .git/worktrees/wt-3/rebase-merge/interactive && + + cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && + refs/heads/fake-3 + $(git rev-parse HEAD~1) + $(git rev-parse HEAD) + refs/heads/fake-4 + $(git rev-parse HEAD) + $(git rev-parse HEAD) + EOF + + for i in 3 4 + do + test_must_fail git branch -f fake-$i HEAD 2>err && + grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || + return 1 + done +' + test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: checked out' ' test_must_fail git fetch server +refs/heads/wt-3:refs/heads/wt-3 2>err && grep "refusing to fetch into branch '\''refs/heads/wt-3'\''" err && -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v4 03/12] branch: consider refs under 'update-refs' 2022-07-12 13:06 ` [PATCH v4 03/12] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget @ 2022-07-15 15:37 ` Phillip Wood 0 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-07-15 15:37 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee On 12/07/2022 14:06, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > The branch_checked_out() helper helps commands like 'git branch' and > 'git fetch' from overwriting refs that are currently checked out in > other worktrees. > > A future update to 'git rebase' will introduce a new '--update-refs' > option which will update the local refs that point to commits that are > being rebased. To avoid collisions as the rebase completes, we want to > make the future data store for these refs to be considered by > branch_checked_out(). > > The data store is a plaintext file inside the 'rebase-merge' directory > for that worktree. The file lists refnames followed by two OIDs, each on > separate lines. The OIDs will be used to store the original values of > the refs and the to-be-written values as the rebase progresses, but can > be ignored at the moment. > > Create a new sequencer_get_update_refs_state() method that parses this > file and populates a struct string_list with the ref-OID pairs. We can > then use this list to add to the current_checked_out_branches strmap > used by branch_checked_out(). > > To properly navigate to the rebase directory for a given worktree, > extract the static strbuf_worktree_gitdir() method to a public API > method. > > We can test that this works without having Git write this file by > artificially creating one in our test script, at least until 'git rebase > --update-refs' is implemented and we can use it directly. This looks good, I've left a couple of comments below but I'm not sure they're even worth reading let alone replying to! Best Wishes Phillip > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > branch.c | 13 +++++++ > sequencer.c | 74 +++++++++++++++++++++++++++++++++++++++ > sequencer.h | 9 +++++ > t/t2407-worktree-heads.sh | 23 ++++++++++++ > 4 files changed, 119 insertions(+) > > diff --git a/branch.c b/branch.c > index 526e8237673..f252c4eef6c 100644 > --- a/branch.c > +++ b/branch.c > @@ -365,6 +365,7 @@ static void prepare_checked_out_branches(void) > char *old; > struct wt_status_state state = { 0 }; > struct worktree *wt = worktrees[i++]; > + struct string_list update_refs = STRING_LIST_INIT_DUP; > > if (wt->is_bare) > continue; > @@ -400,6 +401,18 @@ static void prepare_checked_out_branches(void) > strbuf_release(&ref); > } > wt_status_state_free_buffers(&state); > + > + if (!sequencer_get_update_refs_state(get_worktree_git_dir(wt), > + &update_refs)) { > + struct string_list_item *item; > + for_each_string_list_item(item, &update_refs) { > + old = strmap_put(¤t_checked_out_branches, > + item->string, > + xstrdup(wt->path)); > + free(old); > + } > + string_list_clear(&update_refs, 1); > + } > } > > free_worktrees(worktrees); > diff --git a/sequencer.c b/sequencer.c > index 8c3ed3532ac..e93c61cbd25 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -147,6 +147,20 @@ static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") > */ > static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") > > +/* > + * The update-refs file stores a list of refs that will be updated at the end > + * of the rebase sequence. The 'update-ref <ref>' commands in the todo file > + * update the OIDs for the refs in this file, but the refs are not updated > + * until the end of the rebase sequence. > + * > + * rebase_path_update_refs() returns the path to this file for a given > + * worktree directory. For the current worktree, pass the_repository->gitdir. > + */ > +static char *rebase_path_update_refs(const char *wt_dir) It is nice to see this function. The naming of wt_dir confused me slightly, wt_git_dir would make it clear that this function takes the git_dir of the worktree and not the worktree directory. Theu documentation does a good job of clarifying that though. > +{ > + return xstrfmt("%s/rebase-merge/update-refs", wt_dir); > +} > + > /* > * The following files are written by git-rebase just after parsing the > * command-line. > @@ -169,6 +183,15 @@ static GIT_PATH_FUNC(rebase_path_no_reschedule_failed_exec, "rebase-merge/no-res > static GIT_PATH_FUNC(rebase_path_drop_redundant_commits, "rebase-merge/drop_redundant_commits") > static GIT_PATH_FUNC(rebase_path_keep_redundant_commits, "rebase-merge/keep_redundant_commits") > > +/** > + * A 'struct update_refs_record' represents a value in the update-refs > + * list. We use a string_list to map refs to these (before, after) pairs. > + */ > +struct update_ref_record { > + struct object_id before; > + struct object_id after; > +}; > + > static int git_sequencer_config(const char *k, const char *v, void *cb) > { > struct replay_opts *opts = cb; > @@ -5901,3 +5924,54 @@ int sequencer_determine_whence(struct repository *r, enum commit_whence *whence) > > return 0; > } > + > +int sequencer_get_update_refs_state(const char *wt_dir, > + struct string_list *refs) > +{ > + int result = 0; > + FILE *fp = NULL; > + struct strbuf ref = STRBUF_INIT; > + struct strbuf hash = STRBUF_INIT; > + struct update_ref_record *rec = NULL; > + > + char *path = rebase_path_update_refs(wt_dir); > + > + fp = fopen(path, "r"); > + if (!fp) > + goto cleanup; > + > + while (strbuf_getline(&ref, fp) != EOF) { > + struct string_list_item *item; > + > + CALLOC_ARRAY(rec, 1); It does not matter but later in the series we use rec = xmalloc(sizeof(*rec)) to do this. I'm not sure the project has a standard way of allocating a single instance of something. > + > + if (strbuf_getline(&hash, fp) == EOF || > + get_oid_hex(hash.buf, &rec->before)) { > + warning(_("update-refs file at '%s' is invalid"), > + path); > + result = -1; > + goto cleanup; > + } > + > + if (strbuf_getline(&hash, fp) == EOF || > + get_oid_hex(hash.buf, &rec->after)) { > + warning(_("update-refs file at '%s' is invalid"), > + path); > + result = -1; > + goto cleanup; > + } > + > + item = string_list_insert(refs, ref.buf); > + item->util = rec; > + rec = NULL; > + } > + > +cleanup: > + if (fp) > + fclose(fp); > + free(path); > + free(rec); > + strbuf_release(&ref); > + strbuf_release(&hash); > + return result; > +} > diff --git a/sequencer.h b/sequencer.h > index da64473636b..3ae541bb145 100644 > --- a/sequencer.h > +++ b/sequencer.h > @@ -232,4 +232,13 @@ void sequencer_post_commit_cleanup(struct repository *r, int verbose); > int sequencer_get_last_command(struct repository* r, > enum replay_action *action); > int sequencer_determine_whence(struct repository *r, enum commit_whence *whence); > + > +/** > + * Append the set of ref-OID pairs that are currently stored for the 'git > + * rebase --update-refs' feature if such a rebase is currently happening. > + * > + * Localized to a worktree's git dir. > + */ > +int sequencer_get_update_refs_state(const char *wt_dir, struct string_list *refs); > + > #endif /* SEQUENCER_H */ > diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh > index a67ce5fb003..97f5c87f8c8 100755 > --- a/t/t2407-worktree-heads.sh > +++ b/t/t2407-worktree-heads.sh > @@ -81,6 +81,29 @@ test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer > grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err > ' > > +test_expect_success 'refuse to overwrite: worktree in rebase with --update-refs' ' > + test_when_finished rm -rf .git/worktrees/wt-3/rebase-merge && > + > + mkdir -p .git/worktrees/wt-3/rebase-merge && > + touch .git/worktrees/wt-3/rebase-merge/interactive && > + > + cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && > + refs/heads/fake-3 > + $(git rev-parse HEAD~1) > + $(git rev-parse HEAD) > + refs/heads/fake-4 > + $(git rev-parse HEAD) > + $(git rev-parse HEAD) > + EOF > + > + for i in 3 4 > + do > + test_must_fail git branch -f fake-$i HEAD 2>err && > + grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || > + return 1 > + done > +' > + > test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: checked out' ' > test_must_fail git fetch server +refs/heads/wt-3:refs/heads/wt-3 2>err && > grep "refusing to fetch into branch '\''refs/heads/wt-3'\''" err && ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v4 04/12] rebase-interactive: update 'merge' description 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (2 preceding siblings ...) 2022-07-12 13:06 ` [PATCH v4 03/12] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 ` Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 05/12] sequencer: define array with enum values Derrick Stolee via GitGitGadget ` (10 subsequent siblings) 14 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The 'merge' command description for the todo list documentation in an interactive rebase has multiple lines. The lines other than the first one start with dots ('.') while the similar multi-line documentation for 'fixup' does not. This description only appears in the comment text of the todo file during an interactive rebase. The 'merge' command was documented when interactive rebase was first ported to C in 145e05ac44b (rebase -i: rewrite append_todo_help() in C, 2018-08-10). These dots might have been carried over from the previous shell implementation. The 'fixup' command was documented more recently in 9e3cebd97cb (rebase -i: add fixup [-C | -c] command, 2021-01-29). Looking at the output in an editor, my personal opinion is that the dots are unnecessary and noisy. Remove them now before adding more commands with multi-line documentation. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- rebase-interactive.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rebase-interactive.c b/rebase-interactive.c index 87649d0c016..22394224faa 100644 --- a/rebase-interactive.c +++ b/rebase-interactive.c @@ -54,9 +54,9 @@ void append_todo_help(int command_count, "l, label <label> = label current HEAD with a name\n" "t, reset <label> = reset HEAD to a label\n" "m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]\n" -". create a merge commit using the original merge commit's\n" -". message (or the oneline, if no original merge commit was\n" -". specified); use -c <commit> to reword the commit message\n" +" create a merge commit using the original merge commit's\n" +" message (or the oneline, if no original merge commit was\n" +" specified); use -c <commit> to reword the commit message\n" "\n" "These lines can be re-ordered; they are executed from top to bottom.\n"); unsigned edit_todo = !(shortrevisions && shortonto); -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v4 05/12] sequencer: define array with enum values 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (3 preceding siblings ...) 2022-07-12 13:06 ` [PATCH v4 04/12] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 ` Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 06/12] sequencer: add update-ref command Derrick Stolee via GitGitGadget ` (9 subsequent siblings) 14 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The todo_command_info array defines which strings match with which todo_command enum values. The array is defined in the same order as the enum values, but if one changed without the other, then we would have unexpected results. Make it easier to see changes to the enum and this array by using the enum values as the indices of the array. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/sequencer.c b/sequencer.c index e93c61cbd25..575f9be5ea0 100644 --- a/sequencer.c +++ b/sequencer.c @@ -1708,20 +1708,20 @@ static struct { char c; const char *str; } todo_command_info[] = { - { 'p', "pick" }, - { 0, "revert" }, - { 'e', "edit" }, - { 'r', "reword" }, - { 'f', "fixup" }, - { 's', "squash" }, - { 'x', "exec" }, - { 'b', "break" }, - { 'l', "label" }, - { 't', "reset" }, - { 'm', "merge" }, - { 0, "noop" }, - { 'd', "drop" }, - { 0, NULL } + [TODO_PICK] = { 'p', "pick" }, + [TODO_REVERT] = { 0, "revert" }, + [TODO_EDIT] = { 'e', "edit" }, + [TODO_REWORD] = { 'r', "reword" }, + [TODO_FIXUP] = { 'f', "fixup" }, + [TODO_SQUASH] = { 's', "squash" }, + [TODO_EXEC] = { 'x', "exec" }, + [TODO_BREAK] = { 'b', "break" }, + [TODO_LABEL] = { 'l', "label" }, + [TODO_RESET] = { 't', "reset" }, + [TODO_MERGE] = { 'm', "merge" }, + [TODO_NOOP] = { 0, "noop" }, + [TODO_DROP] = { 'd', "drop" }, + [TODO_COMMENT] = { 0, NULL }, }; static const char *command_to_string(const enum todo_command command) -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v4 06/12] sequencer: add update-ref command 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (4 preceding siblings ...) 2022-07-12 13:06 ` [PATCH v4 05/12] sequencer: define array with enum values Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 ` Derrick Stolee via GitGitGadget 2022-07-12 13:07 ` [PATCH v4 07/12] rebase: add --update-refs option Derrick Stolee via GitGitGadget ` (8 subsequent siblings) 14 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:06 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> Add the boilerplate for an "update-ref" command in the sequencer. This connects to the current no-op do_update_ref() which will be filled in after more connections are created. The syntax in the todo list will be "update-ref <ref-name>" to signal that we should store the current commit as the value for updating <ref-name> at the end of the rebase. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- rebase-interactive.c | 3 +++ sequencer.c | 14 +++++++++++++- sequencer.h | 1 + 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/rebase-interactive.c b/rebase-interactive.c index 22394224faa..1ff07647af3 100644 --- a/rebase-interactive.c +++ b/rebase-interactive.c @@ -57,6 +57,9 @@ void append_todo_help(int command_count, " create a merge commit using the original merge commit's\n" " message (or the oneline, if no original merge commit was\n" " specified); use -c <commit> to reword the commit message\n" +"u, update-ref <ref> = track a placeholder for the <ref> to be updated\n" +" to this position in the new commits. The <ref> is\n" +" updated at the end of the rebase\n" "\n" "These lines can be re-ordered; they are executed from top to bottom.\n"); unsigned edit_todo = !(shortrevisions && shortonto); diff --git a/sequencer.c b/sequencer.c index 575f9be5ea0..1d6442c9639 100644 --- a/sequencer.c +++ b/sequencer.c @@ -1719,6 +1719,7 @@ static struct { [TODO_LABEL] = { 'l', "label" }, [TODO_RESET] = { 't', "reset" }, [TODO_MERGE] = { 'm', "merge" }, + [TODO_UPDATE_REF] = { 'u', "update-ref" }, [TODO_NOOP] = { 0, "noop" }, [TODO_DROP] = { 'd', "drop" }, [TODO_COMMENT] = { 0, NULL }, @@ -2480,7 +2481,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item, command_to_string(item->command)); if (item->command == TODO_EXEC || item->command == TODO_LABEL || - item->command == TODO_RESET) { + item->command == TODO_RESET || item->command == TODO_UPDATE_REF) { item->commit = NULL; item->arg_offset = bol - buf; item->arg_len = (int)(eol - bol); @@ -4079,6 +4080,11 @@ leave_merge: return ret; } +static int do_update_ref(struct repository *r, const char *ref_name) +{ + return 0; +} + static int is_final_fixup(struct todo_list *todo_list) { int i = todo_list->current; @@ -4454,6 +4460,12 @@ static int pick_commits(struct repository *r, return error_with_patch(r, item->commit, arg, item->arg_len, opts, res, 0); + } else if (item->command == TODO_UPDATE_REF) { + struct strbuf ref = STRBUF_INIT; + strbuf_add(&ref, arg, item->arg_len); + if ((res = do_update_ref(r, ref.buf))) + reschedule = 1; + strbuf_release(&ref); } else if (!is_noop(item->command)) return error(_("unknown command %d"), item->command); diff --git a/sequencer.h b/sequencer.h index 3ae541bb145..2cf5c1b9a38 100644 --- a/sequencer.h +++ b/sequencer.h @@ -95,6 +95,7 @@ enum todo_command { TODO_LABEL, TODO_RESET, TODO_MERGE, + TODO_UPDATE_REF, /* commands that do nothing but are counted for reporting progress */ TODO_NOOP, TODO_DROP, -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v4 07/12] rebase: add --update-refs option 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (5 preceding siblings ...) 2022-07-12 13:06 ` [PATCH v4 06/12] sequencer: add update-ref command Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 ` Derrick Stolee via GitGitGadget 2022-07-16 19:30 ` Elijah Newren 2022-07-18 9:05 ` SZEDER Gábor 2022-07-12 13:07 ` [PATCH v4 08/12] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget ` (7 subsequent siblings) 14 siblings, 2 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> When working on a large feature, it can be helpful to break that feature into multiple smaller parts that become reviewed in sequence. During development or during review, a change to one part of the feature could affect multiple of these parts. An interactive rebase can help adjust the multi-part "story" of the branch. However, if there are branches tracking the different parts of the feature, then rebasing the entire list of commits can create commits not reachable from those "sub branches". It can take a manual step to update those branches. Add a new --update-refs option to 'git rebase -i' that adds 'update-ref <ref>' steps to the todo file whenever a commit that is being rebased is decorated with that <ref>. At the very end, the rebase process updates all of the listed refs to the values stored during the rebase operation. Be sure to iterate after any squashing or fixups are placed. Update the branch only after those squashes and fixups are complete. This allows a --fixup commit at the tip of the feature to apply correctly to the sub branch, even if it is fixing up the most-recent commit in that part. One potential problem here is that refs decorating commits that are already marked as "fixup!" or "squash!" will not be included in this list. Generally, the reordering of the "fixup!" and "squash!" is likely to change the relative order of these refs, so it is not recommended. The workflow here is intended to allow these kinds of commits at the tip of the rebased branch while the other sub branches come along for the ride without intervention. This change update the documentation and builtin to accept the --update-refs option as well as updating the todo file with the 'update-ref' commands. Tests are added to ensure that these todo commands are added in the correct locations. This change does _not_ include the actual behavior of tracking the updated refs and writing the new ref values at the end of the rebase process. That is deferred to a later change. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- Documentation/git-rebase.txt | 8 +++ builtin/rebase.c | 5 ++ sequencer.c | 107 ++++++++++++++++++++++++++++++++++ sequencer.h | 1 + t/t2407-worktree-heads.sh | 22 +++++++ t/t3404-rebase-interactive.sh | 70 ++++++++++++++++++++++ 6 files changed, 213 insertions(+) diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index 262fb01aec0..e7611b4089c 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -609,6 +609,13 @@ provided. Otherwise an explicit `--no-reschedule-failed-exec` at the start would be overridden by the presence of `rebase.rescheduleFailedExec=true` configuration. +--update-refs:: +--no-update-refs:: + Automatically force-update any branches that point to commits that + are being rebased. Any branches that are checked out in a worktree + or point to a `squash! ...` or `fixup! ...` commit are not updated + in this way. + INCOMPATIBLE OPTIONS -------------------- @@ -632,6 +639,7 @@ are incompatible with the following options: * --empty= * --reapply-cherry-picks * --edit-todo + * --update-refs * --root when used in combination with --onto In addition, the following pairs of options are incompatible: diff --git a/builtin/rebase.c b/builtin/rebase.c index 7ab50cda2ad..56d82a52106 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -102,6 +102,7 @@ struct rebase_options { int reschedule_failed_exec; int reapply_cherry_picks; int fork_point; + int update_refs; }; #define REBASE_OPTIONS_INIT { \ @@ -298,6 +299,7 @@ static int do_interactive_rebase(struct rebase_options *opts, unsigned flags) ret = complete_action(the_repository, &replay, flags, shortrevisions, opts->onto_name, opts->onto, &opts->orig_head, &commands, opts->autosquash, + opts->update_refs, &todo_list); } @@ -1124,6 +1126,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) OPT_BOOL(0, "autosquash", &options.autosquash, N_("move commits that begin with " "squash!/fixup! under -i")), + OPT_BOOL(0, "update-refs", &options.update_refs, + N_("update local refs that point to commits " + "that are being rebased")), { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"), N_("GPG-sign commits"), PARSE_OPT_OPTARG, NULL, (intptr_t) "" }, diff --git a/sequencer.c b/sequencer.c index 1d6442c9639..e657862cda2 100644 --- a/sequencer.c +++ b/sequencer.c @@ -35,6 +35,7 @@ #include "commit-reach.h" #include "rebase-interactive.h" #include "reset.h" +#include "branch.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -5638,10 +5639,113 @@ static int skip_unnecessary_picks(struct repository *r, return 0; } +struct todo_add_branch_context { + struct todo_item *items; + size_t items_nr; + size_t items_alloc; + struct strbuf *buf; + struct commit *commit; + struct string_list refs_to_oids; +}; + +static int add_decorations_to_list(const struct commit *commit, + struct todo_add_branch_context *ctx) +{ + const struct name_decoration *decoration = get_name_decoration(&commit->object); + + while (decoration) { + struct todo_item *item; + const char *path; + size_t base_offset = ctx->buf->len; + + ALLOC_GROW(ctx->items, + ctx->items_nr + 1, + ctx->items_alloc); + item = &ctx->items[ctx->items_nr]; + memset(item, 0, sizeof(*item)); + + /* If the branch is checked out, then leave a comment instead. */ + if ((path = branch_checked_out(decoration->name))) { + item->command = TODO_COMMENT; + strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", + decoration->name, path); + } else { + struct string_list_item *sti; + item->command = TODO_UPDATE_REF; + strbuf_addf(ctx->buf, "%s\n", decoration->name); + + sti = string_list_insert(&ctx->refs_to_oids, + decoration->name); + sti->util = oiddup(the_hash_algo->null_oid); + } + + item->offset_in_buf = base_offset; + item->arg_offset = base_offset; + item->arg_len = ctx->buf->len - base_offset; + ctx->items_nr++; + + decoration = decoration->next; + } + + return 0; +} + +/* + * For each 'pick' command, find out if the commit has a decoration in + * refs/heads/. If so, then add a 'label for-update-refs/' command. + */ +static int todo_list_add_update_ref_commands(struct todo_list *todo_list) +{ + int i; + static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP; + struct decoration_filter decoration_filter = { + .include_ref_pattern = &decorate_refs_include, + .exclude_ref_pattern = &decorate_refs_exclude, + .exclude_ref_config_pattern = &decorate_refs_exclude_config, + }; + struct todo_add_branch_context ctx = { + .buf = &todo_list->buf, + .refs_to_oids = STRING_LIST_INIT_DUP, + }; + + ctx.items_alloc = 2 * todo_list->nr + 1; + ALLOC_ARRAY(ctx.items, ctx.items_alloc); + + string_list_append(&decorate_refs_include, "refs/heads/"); + load_ref_decorations(&decoration_filter, 0); + + for (i = 0; i < todo_list->nr; ) { + struct todo_item *item = &todo_list->items[i]; + + /* insert ith item into new list */ + ALLOC_GROW(ctx.items, + ctx.items_nr + 1, + ctx.items_alloc); + + ctx.items[ctx.items_nr++] = todo_list->items[i++]; + + if (item->commit) { + ctx.commit = item->commit; + add_decorations_to_list(item->commit, &ctx); + } + } + + string_list_clear(&ctx.refs_to_oids, 1); + free(todo_list->items); + todo_list->items = ctx.items; + todo_list->nr = ctx.items_nr; + todo_list->alloc = ctx.items_alloc; + + return 0; +} + int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags, const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, + unsigned update_refs, struct todo_list *todo_list) { char shortonto[GIT_MAX_HEXSZ + 1]; @@ -5660,6 +5764,9 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla item->arg_len = item->arg_offset = item->flags = item->offset_in_buf = 0; } + if (update_refs && todo_list_add_update_ref_commands(todo_list)) + return -1; + if (autosquash && todo_list_rearrange_squash(todo_list)) return -1; diff --git a/sequencer.h b/sequencer.h index 2cf5c1b9a38..e671d7e0743 100644 --- a/sequencer.h +++ b/sequencer.h @@ -167,6 +167,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, + unsigned update_refs, struct todo_list *todo_list); int todo_list_rearrange_squash(struct todo_list *todo_list); diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index 97f5c87f8c8..8a03f14df8d 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -164,4 +164,26 @@ test_expect_success 'refuse to overwrite when in error states' ' done ' +. "$TEST_DIRECTORY"/lib-rebase.sh + +test_expect_success !SANITIZE_LEAK 'refuse to overwrite during rebase with --update-refs' ' + git commit --fixup HEAD~2 --allow-empty && + ( + set_cat_todo_editor && + test_must_fail git rebase -i --update-refs HEAD~3 >todo && + ! grep "update-refs" todo + ) && + git branch -f allow-update HEAD~2 && + ( + set_cat_todo_editor && + test_must_fail git rebase -i --update-refs HEAD~3 >todo && + grep "update-ref refs/heads/allow-update" todo + ) +' + +# This must be the last test in this file +test_expect_success '$EDITOR and friends are unchanged' ' + test_editor_unchanged +' + test_done diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index f31afd4a547..3cd20733bc8 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1743,6 +1743,76 @@ test_expect_success 'ORIG_HEAD is updated correctly' ' test_cmp_rev ORIG_HEAD test-orig-head@{1} ' +test_expect_success '--update-refs adds label and update-ref commands' ' + git checkout -b update-refs no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git commit --allow-empty --fixup=third && + git branch -f shared-tip && + ( + set_cat_todo_editor && + + cat >expect <<-EOF && + pick $(git log -1 --format=%h J) J + update-ref refs/heads/second + update-ref refs/heads/first + pick $(git log -1 --format=%h K) K + pick $(git log -1 --format=%h L) L + fixup $(git log -1 --format=%h update-refs) fixup! L # empty + update-ref refs/heads/third + pick $(git log -1 --format=%h M) M + update-ref refs/heads/no-conflict-branch + update-ref refs/heads/shared-tip + EOF + + test_must_fail git rebase -i --autosquash --update-refs primary >todo && + test_cmp expect todo + ) +' + +test_expect_success '--update-refs adds commands with --rebase-merges' ' + git checkout -b update-refs-with-merge no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git merge -m merge branch2 && + git branch -f merge-branch && + git commit --fixup=third --allow-empty && + ( + set_cat_todo_editor && + + cat >expect <<-EOF && + label onto + reset onto + pick $(git log -1 --format=%h branch2~1) F + pick $(git log -1 --format=%h branch2) I + update-ref refs/heads/branch2 + label merge + reset onto + pick $(git log -1 --format=%h refs/heads/second) J + update-ref refs/heads/second + update-ref refs/heads/first + pick $(git log -1 --format=%h refs/heads/third~1) K + pick $(git log -1 --format=%h refs/heads/third) L + fixup $(git log -1 --format=%h update-refs-with-merge) fixup! L # empty + update-ref refs/heads/third + pick $(git log -1 --format=%h HEAD~2) M + update-ref refs/heads/no-conflict-branch + merge -C $(git log -1 --format=%h HEAD~1) merge # merge + update-ref refs/heads/merge-branch + EOF + + test_must_fail git rebase -i --autosquash \ + --rebase-merges=rebase-cousins \ + --update-refs primary >todo && + + test_cmp expect todo + ) +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v4 07/12] rebase: add --update-refs option 2022-07-12 13:07 ` [PATCH v4 07/12] rebase: add --update-refs option Derrick Stolee via GitGitGadget @ 2022-07-16 19:30 ` Elijah Newren 2022-07-19 15:50 ` Derrick Stolee 2022-07-18 9:05 ` SZEDER Gábor 1 sibling, 1 reply; 144+ messages in thread From: Elijah Newren @ 2022-07-16 19:30 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood, Derrick Stolee On Tue, Jul 12, 2022 at 6:07 AM Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com> wrote: > > From: Derrick Stolee <derrickstolee@github.com> > [...] > > +--update-refs:: > +--no-update-refs:: > + Automatically force-update any branches that point to commits that > + are being rebased. Any branches that are checked out in a worktree > + or point to a `squash! ...` or `fixup! ...` commit are not updated > + in this way. I think the second sentence here should be split. In particular, I don't think I understand the second half of the second sentence. Do you intend to say here that branches pointing to a `squash!` or `fixup!` will instead update the first `pick` in the ancestry of such a commit, rather than that such branches are entirely excluded from any updates? That's what I observed in my testing of your v3, at least, and that's the behavior I'd expect this feature to implement, but this documentation doesn't match. [...] > @@ -5660,6 +5764,9 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla > item->arg_len = item->arg_offset = item->flags = item->offset_in_buf = 0; > } > > + if (update_refs && todo_list_add_update_ref_commands(todo_list)) > + return -1; > + As a tangent, I find it interesting that you add the update-ref commands as a post-processing step rather than as a part of sequencer_make_script(). I don't think you need to change anything, but I am curious due to my git-replay work if you find it advantageous to do it this way. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 07/12] rebase: add --update-refs option 2022-07-16 19:30 ` Elijah Newren @ 2022-07-19 15:50 ` Derrick Stolee 0 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee @ 2022-07-19 15:50 UTC (permalink / raw) To: Elijah Newren, Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood On 7/16/2022 3:30 PM, Elijah Newren wrote: > On Tue, Jul 12, 2022 at 6:07 AM Derrick Stolee via GitGitGadget > <gitgitgadget@gmail.com> wrote: >> >> From: Derrick Stolee <derrickstolee@github.com> >> > [...] >> >> +--update-refs:: >> +--no-update-refs:: >> + Automatically force-update any branches that point to commits that >> + are being rebased. Any branches that are checked out in a worktree >> + or point to a `squash! ...` or `fixup! ...` commit are not updated >> + in this way. > > I think the second sentence here should be split. In particular, I > don't think I understand the second half of the second sentence. Do > you intend to say here that branches pointing to a `squash!` or > `fixup!` will instead update the first `pick` in the ancestry of such > a commit, rather than that such branches are entirely excluded from > any updates? That's what I observed in my testing of your v3, at > least, and that's the behavior I'd expect this feature to implement, > but this documentation doesn't match. Good find. You're right that these don't match, and its in fact that I expected it to work this way, but it doesn't. I've added a branch to my tests that points to a fixup! that is not the tip commit and use it to demonstrate that it is not reordered, but _does_ appear in the 'update-ref <ref>' list. I'll update the documentation to match this behavior, too. This case is unlikely to happen much in practice, but I now believe it's better to include the ref than to ignore it. > [...] >> @@ -5660,6 +5764,9 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla >> item->arg_len = item->arg_offset = item->flags = item->offset_in_buf = 0; >> } >> >> + if (update_refs && todo_list_add_update_ref_commands(todo_list)) >> + return -1; >> + > > As a tangent, I find it interesting that you add the update-ref > commands as a post-processing step rather than as a part of > sequencer_make_script(). I don't think you need to change anything, > but I am curious due to my git-replay work if you find it advantageous > to do it this way. I found it to be simple enough to do a scan of the steps directly instead of injecting extra logic into the _make_script() method. The simplest reason is that we would need to inject "update_refs" into that method. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 07/12] rebase: add --update-refs option 2022-07-12 13:07 ` [PATCH v4 07/12] rebase: add --update-refs option Derrick Stolee via GitGitGadget 2022-07-16 19:30 ` Elijah Newren @ 2022-07-18 9:05 ` SZEDER Gábor 2022-07-18 16:55 ` Derrick Stolee 1 sibling, 1 reply; 144+ messages in thread From: SZEDER Gábor @ 2022-07-18 9:05 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee On Tue, Jul 12, 2022 at 01:07:00PM +0000, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > When working on a large feature, it can be helpful to break that feature > into multiple smaller parts that become reviewed in sequence. During > development or during review, a change to one part of the feature could > affect multiple of these parts. An interactive rebase can help adjust > the multi-part "story" of the branch. > > However, if there are branches tracking the different parts of the > feature, then rebasing the entire list of commits can create commits not > reachable from those "sub branches". It can take a manual step to update > those branches. > > Add a new --update-refs option to 'git rebase -i' that adds 'update-ref > <ref>' steps to the todo file whenever a commit that is being rebased is > decorated with that <ref>. At the very end, the rebase process updates > all of the listed refs to the values stored during the rebase operation. > > Be sure to iterate after any squashing or fixups are placed. Update the > branch only after those squashes and fixups are complete. This allows a > --fixup commit at the tip of the feature to apply correctly to the sub > branch, even if it is fixing up the most-recent commit in that part. > > One potential problem here is that refs decorating commits that are > already marked as "fixup!" or "squash!" will not be included in this > list. Generally, the reordering of the "fixup!" and "squash!" is likely > to change the relative order of these refs, so it is not recommended. > The workflow here is intended to allow these kinds of commits at the tip > of the rebased branch while the other sub branches come along for the > ride without intervention. > > This change update the documentation and builtin to accept the > --update-refs option as well as updating the todo file with the > 'update-ref' commands. Tests are added to ensure that these todo > commands are added in the correct locations. > > This change does _not_ include the actual behavior of tracking the > updated refs and writing the new ref values at the end of the rebase > process. That is deferred to a later change. > > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > Documentation/git-rebase.txt | 8 +++ > builtin/rebase.c | 5 ++ > sequencer.c | 107 ++++++++++++++++++++++++++++++++++ > sequencer.h | 1 + > t/t2407-worktree-heads.sh | 22 +++++++ > t/t3404-rebase-interactive.sh | 70 ++++++++++++++++++++++ > 6 files changed, 213 insertions(+) > > diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt > index 262fb01aec0..e7611b4089c 100644 > --- a/Documentation/git-rebase.txt > +++ b/Documentation/git-rebase.txt > @@ -609,6 +609,13 @@ provided. Otherwise an explicit `--no-reschedule-failed-exec` at the > start would be overridden by the presence of > `rebase.rescheduleFailedExec=true` configuration. > > +--update-refs:: So the option is called '--update-refs', but ... > +--no-update-refs:: > + Automatically force-update any branches that point to commits that ... its description talks about "branches". > + are being rebased. Any branches that are checked out in a worktree > + or point to a `squash! ...` or `fixup! ...` commit are not updated > + in this way. > + > INCOMPATIBLE OPTIONS > -------------------- > > @@ -1124,6 +1126,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) > OPT_BOOL(0, "autosquash", &options.autosquash, > N_("move commits that begin with " > "squash!/fixup! under -i")), > + OPT_BOOL(0, "update-refs", &options.update_refs, > + N_("update local refs that point to commits " And its short help talks about "local refs". I think at least the documentation and short help should use consistent terminology. > + "that are being rebased")), > { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"), > N_("GPG-sign commits"), > PARSE_OPT_OPTARG, NULL, (intptr_t) "" }, ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 07/12] rebase: add --update-refs option 2022-07-18 9:05 ` SZEDER Gábor @ 2022-07-18 16:55 ` Derrick Stolee 2022-07-18 19:35 ` Junio C Hamano 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-07-18 16:55 UTC (permalink / raw) To: SZEDER Gábor, Derrick Stolee via GitGitGadget Cc: git, gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 7/18/2022 5:05 AM, SZEDER Gábor wrote: > On Tue, Jul 12, 2022 at 01:07:00PM +0000, Derrick Stolee via GitGitGadget wrote: >> From: Derrick Stolee <derrickstolee@github.com> >> +--update-refs:: > > So the option is called '--update-refs', but ... > >> +--no-update-refs:: >> + Automatically force-update any branches that point to commits that > > ... its description talks about "branches". >> + OPT_BOOL(0, "update-refs", &options.update_refs, >> + N_("update local refs that point to commits " > > And its short help talks about "local refs". > > I think at least the documentation and short help should use > consistent terminology. Thanks for catching this. I think I should use "branches" here, but keep the name "--update-refs". The biggest reason is that it provides a nice parallel with the "update-ref" sequencer command. This command allows updating _any_ ref, such as lightweight tags in refs/tags/* or even refs in refs/my/namespace/*. The --update-refs option doesn't create the commands to update tags or refs in places other than refs/heads/*. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 07/12] rebase: add --update-refs option 2022-07-18 16:55 ` Derrick Stolee @ 2022-07-18 19:35 ` Junio C Hamano 2022-07-19 15:53 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-07-18 19:35 UTC (permalink / raw) To: Derrick Stolee Cc: SZEDER Gábor, Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren Derrick Stolee <derrickstolee@github.com> writes: > ... I think I should use "branches" here, but > keep the name "--update-refs". The biggest reason is that it provides > a nice parallel with the "update-ref" sequencer command. This command > allows updating _any_ ref, such as lightweight tags in refs/tags/* > or even refs in refs/my/namespace/*. > > The --update-refs option doesn't create the commands to update tags > or refs in places other than refs/heads/*. I guess it would make the choice of "branch" the most appropriate. I was hoping that we can repoint refs in private namespaces that are not branches with the option. But as long as the underlying "update-ref" instruction can be used by advanced users, that is OK. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 07/12] rebase: add --update-refs option 2022-07-18 19:35 ` Junio C Hamano @ 2022-07-19 15:53 ` Derrick Stolee 2022-07-19 16:44 ` Junio C Hamano 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-07-19 15:53 UTC (permalink / raw) To: Junio C Hamano Cc: SZEDER Gábor, Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 7/18/2022 3:35 PM, Junio C Hamano wrote: > Derrick Stolee <derrickstolee@github.com> writes: > >> ... I think I should use "branches" here, but >> keep the name "--update-refs". The biggest reason is that it provides >> a nice parallel with the "update-ref" sequencer command. This command >> allows updating _any_ ref, such as lightweight tags in refs/tags/* >> or even refs in refs/my/namespace/*. >> >> The --update-refs option doesn't create the commands to update tags >> or refs in places other than refs/heads/*. > > I guess it would make the choice of "branch" the most appropriate. > > I was hoping that we can repoint refs in private namespaces that are > not branches with the option. But as long as the underlying > "update-ref" instruction can be used by advanced users, that is OK. I would like to keep the --update-refs name for a couple reasons: 1. 'update-ref' is the right name for the sequencer command. Having a parallel there is helpful for learning about the option. 2. We could extend the boolean '--update-refs' option into a more advanced multi-valued '--update-refs=<refspec>' option to allow advanced users to specify a ref namespace that they want included. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 07/12] rebase: add --update-refs option 2022-07-19 15:53 ` Derrick Stolee @ 2022-07-19 16:44 ` Junio C Hamano 2022-07-19 16:47 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-07-19 16:44 UTC (permalink / raw) To: Derrick Stolee Cc: SZEDER Gábor, Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren Derrick Stolee <derrickstolee@github.com> writes: > On 7/18/2022 3:35 PM, Junio C Hamano wrote: >> Derrick Stolee <derrickstolee@github.com> writes: >> >>> ... I think I should use "branches" here, but >>> keep the name "--update-refs". The biggest reason is that it provides >>> a nice parallel with the "update-ref" sequencer command. This command >>> allows updating _any_ ref, such as lightweight tags in refs/tags/* >>> or even refs in refs/my/namespace/*. >>> >>> The --update-refs option doesn't create the commands to update tags >>> or refs in places other than refs/heads/*. >> >> I guess it would make the choice of "branch" the most appropriate. >> >> I was hoping that we can repoint refs in private namespaces that are >> not branches with the option. But as long as the underlying >> "update-ref" instruction can be used by advanced users, that is OK. > > I would like to keep the --update-refs name for a couple reasons: I do not think anybody proposed to change the name of that option. I was reacting to your "I should use branches here", with the understanding that "here" is this place where you used "local refs". >> + OPT_BOOL(0, "update-refs", &options.update_refs, >> + N_("update local refs that point to commits " If "rebase --update-refs" uses "update-ref" insn (which is capable of repointing non-branch refs) only for local branches, then the help text for the "--update-refs" option can safely say "update local branches" without being inaccurate. That is where my "branch is the most appropriate" comes from. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 07/12] rebase: add --update-refs option 2022-07-19 16:44 ` Junio C Hamano @ 2022-07-19 16:47 ` Derrick Stolee 0 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee @ 2022-07-19 16:47 UTC (permalink / raw) To: Junio C Hamano Cc: SZEDER Gábor, Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren On 7/19/2022 12:44 PM, Junio C Hamano wrote: > Derrick Stolee <derrickstolee@github.com> writes: > >> On 7/18/2022 3:35 PM, Junio C Hamano wrote: >>> Derrick Stolee <derrickstolee@github.com> writes: >>> >>>> ... I think I should use "branches" here, but >>>> keep the name "--update-refs". The biggest reason is that it provides >>>> a nice parallel with the "update-ref" sequencer command. This command >>>> allows updating _any_ ref, such as lightweight tags in refs/tags/* >>>> or even refs in refs/my/namespace/*. >>>> >>>> The --update-refs option doesn't create the commands to update tags >>>> or refs in places other than refs/heads/*. >>> >>> I guess it would make the choice of "branch" the most appropriate. >>> >>> I was hoping that we can repoint refs in private namespaces that are >>> not branches with the option. But as long as the underlying >>> "update-ref" instruction can be used by advanced users, that is OK. >> >> I would like to keep the --update-refs name for a couple reasons: > > I do not think anybody proposed to change the name of that option. > > I was reacting to your "I should use branches here", with the > understanding that "here" is this place where you used "local refs". > >>> + OPT_BOOL(0, "update-refs", &options.update_refs, >>> + N_("update local refs that point to commits " > > If "rebase --update-refs" uses "update-ref" insn (which is capable > of repointing non-branch refs) only for local branches, then the > help text for the "--update-refs" option can safely say "update > local branches" without being inaccurate. That is where my "branch > is the most appropriate" comes from. Thanks for the clarification. Sorry for misunderstanding. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v4 08/12] rebase: update refs from 'update-ref' commands 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (6 preceding siblings ...) 2022-07-12 13:07 ` [PATCH v4 07/12] rebase: add --update-refs option Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 ` Derrick Stolee via GitGitGadget 2022-07-15 13:25 ` Phillip Wood 2022-07-12 13:07 ` [PATCH v4 09/12] sequencer: rewrite update-refs as user edits todo list Derrick Stolee via GitGitGadget ` (6 subsequent siblings) 14 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The previous change introduced the 'git rebase --update-refs' option which added 'update-ref <ref>' commands to the todo list of an interactive rebase. Teach Git to record the HEAD position when reaching these 'update-ref' commands. The ref/before/after triple is stored in the $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this file to avoid having other processes updating the refs in that file while the rebase is in progress. Not only do we update the file when the sequencer reaches these 'update-ref' commands, we then update the refs themselves at the end of the rebase sequence. If the rebase is aborted before this final step, then the refs are not updated. The 'before' value is used to ensure that we do not accidentally obliterate a ref that was updated concurrently (say, by an older version of Git or a third-party tool). Now that the 'git rebase --update-refs' command is implemented to write to the update-refs file, we can remove the fake construction of the update-refs file from a test in t2407-worktree-heads.sh. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 113 +++++++++++++++++++++++++++++++++- t/t2407-worktree-heads.sh | 21 ++----- t/t3404-rebase-interactive.sh | 17 +++++ 3 files changed, 134 insertions(+), 17 deletions(-) diff --git a/sequencer.c b/sequencer.c index e657862cda2..2d89b3b727a 100644 --- a/sequencer.c +++ b/sequencer.c @@ -36,6 +36,7 @@ #include "rebase-interactive.h" #include "reset.h" #include "branch.h" +#include "log-tree.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -193,6 +194,19 @@ struct update_ref_record { struct object_id after; }; +static struct update_ref_record *init_update_ref_record(const char *ref) +{ + struct update_ref_record *rec = xmalloc(sizeof(*rec)); + + oidcpy(&rec->before, null_oid()); + oidcpy(&rec->after, null_oid()); + + /* This may fail, but that's fine, we will keep the null OID. */ + read_ref(ref, &rec->before); + + return rec; +} + static int git_sequencer_config(const char *k, const char *v, void *cb) { struct replay_opts *opts = cb; @@ -4081,11 +4095,102 @@ leave_merge: return ret; } -static int do_update_ref(struct repository *r, const char *ref_name) +static int write_update_refs_state(struct string_list *refs_to_oids) { + int result = 0; + struct lock_file lock = LOCK_INIT; + FILE *fp = NULL; + struct string_list_item *item; + char *path; + + if (!refs_to_oids->nr) + return 0; + + path = rebase_path_update_refs(the_repository->gitdir); + + if (safe_create_leading_directories(path)) { + result = error(_("unable to create leading directories of %s"), + path); + goto cleanup; + } + + if (hold_lock_file_for_update(&lock, path, 0) < 0) { + result = error(_("another 'rebase' process appears to be running; " + "'%s.lock' already exists"), + path); + goto cleanup; + } + + fp = fdopen_lock_file(&lock, "w"); + if (!fp) { + result = error_errno(_("could not open '%s' for writing"), path); + rollback_lock_file(&lock); + goto cleanup; + } + + for_each_string_list_item(item, refs_to_oids) { + struct update_ref_record *rec = item->util; + fprintf(fp, "%s\n%s\n%s\n", item->string, + oid_to_hex(&rec->before), oid_to_hex(&rec->after)); + } + + result = commit_lock_file(&lock); + +cleanup: + free(path); + return result; +} + +static int do_update_ref(struct repository *r, const char *refname) +{ + struct string_list_item *item; + struct string_list list = STRING_LIST_INIT_DUP; + + sequencer_get_update_refs_state(r->gitdir, &list); + + for_each_string_list_item(item, &list) { + if (!strcmp(item->string, refname)) { + struct update_ref_record *rec = item->util; + read_ref("HEAD", &rec->after); + break; + } + } + + write_update_refs_state(&list); + string_list_clear(&list, 1); return 0; } +static int do_update_refs(struct repository *r) +{ + int res = 0; + struct string_list_item *item; + struct string_list refs_to_oids = STRING_LIST_INIT_DUP; + struct ref_store *refs = get_main_ref_store(r); + + sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); + + for_each_string_list_item(item, &refs_to_oids) { + struct update_ref_record *rec = item->util; + + if (oideq(&rec->after, the_hash_algo->null_oid)) { + /* + * Ref was not updated. User may have deleted the + * 'update-ref' step. + */ + continue; + } + + res |= refs_update_ref(refs, "rewritten during rebase", + item->string, + &rec->after, &rec->before, + 0, UPDATE_REFS_MSG_ON_ERR); + } + + string_list_clear(&refs_to_oids, 1); + return res; +} + static int is_final_fixup(struct todo_list *todo_list) { int i = todo_list->current; @@ -4603,6 +4708,8 @@ cleanup_head_ref: strbuf_release(&head_ref); } + do_update_refs(r); + /* * Sequence of picks finished successfully; cleanup by * removing the .git/sequencer directory @@ -5676,7 +5783,7 @@ static int add_decorations_to_list(const struct commit *commit, sti = string_list_insert(&ctx->refs_to_oids, decoration->name); - sti->util = oiddup(the_hash_algo->null_oid); + sti->util = init_update_ref_record(decoration->name); } item->offset_in_buf = base_offset; @@ -5732,6 +5839,8 @@ static int todo_list_add_update_ref_commands(struct todo_list *todo_list) } } + write_update_refs_state(&ctx.refs_to_oids); + string_list_clear(&ctx.refs_to_oids, 1); free(todo_list->items); todo_list->items = ctx.items; diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index 8a03f14df8d..50815acd3e8 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -81,25 +81,16 @@ test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err ' -test_expect_success 'refuse to overwrite: worktree in rebase with --update-refs' ' - test_when_finished rm -rf .git/worktrees/wt-3/rebase-merge && - - mkdir -p .git/worktrees/wt-3/rebase-merge && - touch .git/worktrees/wt-3/rebase-merge/interactive && +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase with --update-refs' ' + test_when_finished git -C wt-3 rebase --abort && - cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && - refs/heads/fake-3 - $(git rev-parse HEAD~1) - $(git rev-parse HEAD) - refs/heads/fake-4 - $(git rev-parse HEAD) - $(git rev-parse HEAD) - EOF + git branch -f can-be-updated wt-3 && + test_must_fail git -C wt-3 rebase --update-refs conflict-3 && for i in 3 4 do - test_must_fail git branch -f fake-$i HEAD 2>err && - grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || + test_must_fail git branch -f can-be-updated HEAD 2>err && + grep "cannot force update the branch '\''can-be-updated'\'' checked out at.*wt-3" err || return 1 done ' diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 3cd20733bc8..a37820fa728 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1813,6 +1813,23 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' ) ' +test_expect_success '--update-refs updates refs correctly' ' + git checkout -B update-refs no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + test_commit extra2 fileX && + git commit --amend --fixup=L && + + git rebase -i --autosquash --update-refs primary && + + test_cmp_rev HEAD~3 refs/heads/first && + test_cmp_rev HEAD~3 refs/heads/second && + test_cmp_rev HEAD~1 refs/heads/third && + test_cmp_rev HEAD refs/heads/no-conflict-branch +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v4 08/12] rebase: update refs from 'update-ref' commands 2022-07-12 13:07 ` [PATCH v4 08/12] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget @ 2022-07-15 13:25 ` Phillip Wood 2022-07-19 16:04 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Phillip Wood @ 2022-07-15 13:25 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee Hi Stolee On 12/07/2022 14:07, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > The previous change introduced the 'git rebase --update-refs' option > which added 'update-ref <ref>' commands to the todo list of an > interactive rebase. > > Teach Git to record the HEAD position when reaching these 'update-ref' > commands. The ref/before/after triple is stored in the > $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this > file to avoid having other processes updating the refs in that file > while the rebase is in progress. > > Not only do we update the file when the sequencer reaches these > 'update-ref' commands, we then update the refs themselves at the end of > the rebase sequence. If the rebase is aborted before this final step, > then the refs are not updated. The 'before' value is used to ensure that > we do not accidentally obliterate a ref that was updated concurrently > (say, by an older version of Git or a third-party tool). > > Now that the 'git rebase --update-refs' command is implemented to write > to the update-refs file, we can remove the fake construction of the > update-refs file from a test in t2407-worktree-heads.sh. This is looking good. I've left a few comments, mostly about error propagation. It's nice to see us recording the initial value of the ref when the todo list is created. It's also good to see this using a lock file. We could perhaps lock the file (with a timeout) when we read it in sequencer_get_update_refs_state() to avoid a race where a process is checking out a new branch in one worktree and another is preparing to rebase that branch in another worktree. > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > sequencer.c | 113 +++++++++++++++++++++++++++++++++- > t/t2407-worktree-heads.sh | 21 ++----- > t/t3404-rebase-interactive.sh | 17 +++++ > 3 files changed, 134 insertions(+), 17 deletions(-) > > diff --git a/sequencer.c b/sequencer.c > index e657862cda2..2d89b3b727a 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -36,6 +36,7 @@ > #include "rebase-interactive.h" > #include "reset.h" > #include "branch.h" > +#include "log-tree.h" > > #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" > > @@ -193,6 +194,19 @@ struct update_ref_record { > struct object_id after; > }; > > +static struct update_ref_record *init_update_ref_record(const char *ref) > +{ > + struct update_ref_record *rec = xmalloc(sizeof(*rec)); > + > + oidcpy(&rec->before, null_oid()); > + oidcpy(&rec->after, null_oid()); > + > + /* This may fail, but that's fine, we will keep the null OID. */ > + read_ref(ref, &rec->before); > + > + return rec; > +} > + > static int git_sequencer_config(const char *k, const char *v, void *cb) > { > struct replay_opts *opts = cb; > @@ -4081,11 +4095,102 @@ leave_merge: > return ret; > } > > -static int do_update_ref(struct repository *r, const char *ref_name) > +static int write_update_refs_state(struct string_list *refs_to_oids) > { > + int result = 0; > + struct lock_file lock = LOCK_INIT; > + FILE *fp = NULL; > + struct string_list_item *item; > + char *path; > + > + if (!refs_to_oids->nr) > + return 0; > + > + path = rebase_path_update_refs(the_repository->gitdir); > + > + if (safe_create_leading_directories(path)) { > + result = error(_("unable to create leading directories of %s"), > + path); > + goto cleanup; > + } > + > + if (hold_lock_file_for_update(&lock, path, 0) < 0) { > + result = error(_("another 'rebase' process appears to be running; " > + "'%s.lock' already exists"), > + path); > + goto cleanup; > + } > + > + fp = fdopen_lock_file(&lock, "w"); > + if (!fp) { > + result = error_errno(_("could not open '%s' for writing"), path); > + rollback_lock_file(&lock); > + goto cleanup; > + } > + > + for_each_string_list_item(item, refs_to_oids) { > + struct update_ref_record *rec = item->util; > + fprintf(fp, "%s\n%s\n%s\n", item->string, > + oid_to_hex(&rec->before), oid_to_hex(&rec->after)); > + } > + > + result = commit_lock_file(&lock); > + > +cleanup: > + free(path); > + return result; > +} > + > +static int do_update_ref(struct repository *r, const char *refname) > +{ > + struct string_list_item *item; > + struct string_list list = STRING_LIST_INIT_DUP; > + > + sequencer_get_update_refs_state(r->gitdir, &list); We're ignoring any errors here and always returning 0 from this function. > + > + for_each_string_list_item(item, &list) { > + if (!strcmp(item->string, refname)) { > + struct update_ref_record *rec = item->util; > + read_ref("HEAD", &rec->after); > + break; > + } > + } > + > + write_update_refs_state(&list); > + string_list_clear(&list, 1); > return 0; > } > > +static int do_update_refs(struct repository *r) > +{ > + int res = 0; > + struct string_list_item *item; > + struct string_list refs_to_oids = STRING_LIST_INIT_DUP; > + struct ref_store *refs = get_main_ref_store(r); > + > + sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); > + > + for_each_string_list_item(item, &refs_to_oids) { > + struct update_ref_record *rec = item->util; > + > + if (oideq(&rec->after, the_hash_algo->null_oid)) { > + /* > + * Ref was not updated. User may have deleted the > + * 'update-ref' step. > + */ Unless we want to support users editing the todo list without using "git rebase --edit-todo" then by the end of the series it is a bug if we leave an entry in the update-refs file that has been removed from the todo list so I wander if we should remove this if(). > + continue; > + } > + > + res |= refs_update_ref(refs, "rewritten during rebase", > + item->string, > + &rec->after, &rec->before, > + 0, UPDATE_REFS_MSG_ON_ERR); > + } > + > + string_list_clear(&refs_to_oids, 1); > + return res; > +} > + > static int is_final_fixup(struct todo_list *todo_list) > { > int i = todo_list->current; > @@ -4603,6 +4708,8 @@ cleanup_head_ref: > strbuf_release(&head_ref); > } > > + do_update_refs(r); Should this be inside the "if (is_rebase_i(opts))" that is closed just above it? We're also ignoring the return value. > + > /* > * Sequence of picks finished successfully; cleanup by > * removing the .git/sequencer directory > @@ -5676,7 +5783,7 @@ static int add_decorations_to_list(const struct commit *commit, > > sti = string_list_insert(&ctx->refs_to_oids, > decoration->name); > - sti->util = oiddup(the_hash_algo->null_oid); > + sti->util = init_update_ref_record(decoration->name); > } > > item->offset_in_buf = base_offset; > @@ -5732,6 +5839,8 @@ static int todo_list_add_update_ref_commands(struct todo_list *todo_list) > } > } > > + write_update_refs_state(&ctx.refs_to_oids); We're ignoring the return value. Also I think todo_list_add_update_ref_commands() only ever returns 0. Best Wishes Phillip > + > string_list_clear(&ctx.refs_to_oids, 1); > free(todo_list->items); > todo_list->items = ctx.items; > diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh > index 8a03f14df8d..50815acd3e8 100755 > --- a/t/t2407-worktree-heads.sh > +++ b/t/t2407-worktree-heads.sh > @@ -81,25 +81,16 @@ test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer > grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err > ' > > -test_expect_success 'refuse to overwrite: worktree in rebase with --update-refs' ' > - test_when_finished rm -rf .git/worktrees/wt-3/rebase-merge && > - > - mkdir -p .git/worktrees/wt-3/rebase-merge && > - touch .git/worktrees/wt-3/rebase-merge/interactive && > +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase with --update-refs' ' > + test_when_finished git -C wt-3 rebase --abort && > > - cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && > - refs/heads/fake-3 > - $(git rev-parse HEAD~1) > - $(git rev-parse HEAD) > - refs/heads/fake-4 > - $(git rev-parse HEAD) > - $(git rev-parse HEAD) > - EOF > + git branch -f can-be-updated wt-3 && > + test_must_fail git -C wt-3 rebase --update-refs conflict-3 && > > for i in 3 4 > do > - test_must_fail git branch -f fake-$i HEAD 2>err && > - grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || > + test_must_fail git branch -f can-be-updated HEAD 2>err && > + grep "cannot force update the branch '\''can-be-updated'\'' checked out at.*wt-3" err || > return 1 > done > ' > diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh > index 3cd20733bc8..a37820fa728 100755 > --- a/t/t3404-rebase-interactive.sh > +++ b/t/t3404-rebase-interactive.sh > @@ -1813,6 +1813,23 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' > ) > ' > > +test_expect_success '--update-refs updates refs correctly' ' > + git checkout -B update-refs no-conflict-branch && > + git branch -f base HEAD~4 && > + git branch -f first HEAD~3 && > + git branch -f second HEAD~3 && > + git branch -f third HEAD~1 && > + test_commit extra2 fileX && > + git commit --amend --fixup=L && > + > + git rebase -i --autosquash --update-refs primary && > + > + test_cmp_rev HEAD~3 refs/heads/first && > + test_cmp_rev HEAD~3 refs/heads/second && > + test_cmp_rev HEAD~1 refs/heads/third && > + test_cmp_rev HEAD refs/heads/no-conflict-branch > +' > + > # This must be the last test in this file > test_expect_success '$EDITOR and friends are unchanged' ' > test_editor_unchanged ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 08/12] rebase: update refs from 'update-ref' commands 2022-07-15 13:25 ` Phillip Wood @ 2022-07-19 16:04 ` Derrick Stolee 0 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee @ 2022-07-19 16:04 UTC (permalink / raw) To: phillip.wood, Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren On 7/15/2022 9:25 AM, Phillip Wood wrote: >> Now that the 'git rebase --update-refs' command is implemented to write >> to the update-refs file, we can remove the fake construction of the >> update-refs file from a test in t2407-worktree-heads.sh. > > This is looking good. I've left a few comments, mostly about error propagation. It's nice to see us recording the initial value of the ref when the todo list is created. It's also good to see this using a lock file. We could perhaps lock the file (with a timeout) when we read it in sequencer_get_update_refs_state() to avoid a race where a process is checking out a new branch in one worktree and another is preparing to rebase that branch in another worktree. >> +static int do_update_ref(struct repository *r, const char *refname) >> +{ >> + struct string_list_item *item; >> + struct string_list list = STRING_LIST_INIT_DUP; >> + >> + sequencer_get_update_refs_state(r->gitdir, &list); > > We're ignoring any errors here and always returning 0 from this function. Thanks. Will fix. >> + >> + for_each_string_list_item(item, &list) { >> + if (!strcmp(item->string, refname)) { >> + struct update_ref_record *rec = item->util; >> + read_ref("HEAD", &rec->after); This is the other place where we could have a failure. >> +static int do_update_refs(struct repository *r) >> +{ >> + int res = 0; >> + struct string_list_item *item; >> + struct string_list refs_to_oids = STRING_LIST_INIT_DUP; >> + struct ref_store *refs = get_main_ref_store(r); >> + >> + sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); We need to check for failure here, too. >> + for_each_string_list_item(item, &refs_to_oids) { >> + struct update_ref_record *rec = item->util; >> + >> + if (oideq(&rec->after, the_hash_algo->null_oid)) { >> + /* >> + * Ref was not updated. User may have deleted the >> + * 'update-ref' step. >> + */ > > Unless we want to support users editing the todo list without using "git rebase --edit-todo" then by the end of the series it is a bug if we leave an entry in the update-refs file that has been removed from the todo list so I wander if we should remove this if(). I think this is leftover from the previous version and will never happen. If rec->after is null, then it would be removed earlier when parsing the todo list. >> + do_update_refs(r); > > Should this be inside the "if (is_rebase_i(opts))" that is closed just above it? We're also ignoring the return value. Couldn't hurt. Should be a no-op if not in interactive mode. >> + write_update_refs_state(&ctx.refs_to_oids); > > We're ignoring the return value. Also I think todo_list_add_update_ref_commands() only ever returns 0. Thanks for your attention to detail here. -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v4 09/12] sequencer: rewrite update-refs as user edits todo list 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (7 preceding siblings ...) 2022-07-12 13:07 ` [PATCH v4 08/12] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 ` Derrick Stolee via GitGitGadget 2022-07-15 10:27 ` Phillip Wood 2022-07-16 19:20 ` Elijah Newren 2022-07-12 13:07 ` [PATCH v4 10/12] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget ` (5 subsequent siblings) 14 siblings, 2 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> An interactive rebase provides opportunities for the user to edit the todo list. The --update-refs option initializes the list with some 'update-ref <ref>' steps, but the user could add these manually. Further, the user could add or remove these steps during pauses in the interactive rebase. Add a new method, todo_list_filter_update_refs(), that scans a todo_list and compares it to the stored update-refs file. There are two actions that can happen at this point: 1. If a '<ref>/<before>/<after>' triple in the update-refs file does not have a matching 'update-ref <ref>' command in the todo-list _and_ the <after> value is the null OID, then remove that triple. Here, the user removed the 'update-ref <ref>' command before it was executed, since if it was executed then the <after> value would store the commit at that position. 2. If a 'update-ref <ref>' command in the todo-list does not have a matching '<ref>/<before>/<after>' triple in the update-refs file, then insert a new one. Store the <before> value to be the current OID pointed at by <ref>. This is handled inside of the init_update_ref_record() helper method. We can test that this works by rewriting the todo-list several times in the course of a rebase. Check that each ref is locked or unlocked for updates after each todo-list update. We an also verify that the ref update fails if a concurrent process updates one of the refs after the rebase process records the "locked" ref location. To help these tests, add a new 'set_replace_editor' helper that will replace the todo-list with an exact file. Reported-by: Phillip Wood <phillip.wood123@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- rebase-interactive.c | 6 ++ sequencer.c | 96 +++++++++++++++++++++++ sequencer.h | 12 +++ t/lib-rebase.sh | 15 ++++ t/t3404-rebase-interactive.sh | 139 ++++++++++++++++++++++++++++++++++ 5 files changed, 268 insertions(+) diff --git a/rebase-interactive.c b/rebase-interactive.c index 1ff07647af3..7407c593191 100644 --- a/rebase-interactive.c +++ b/rebase-interactive.c @@ -146,6 +146,12 @@ int edit_todo_list(struct repository *r, struct todo_list *todo_list, return -4; } + /* + * See if branches need to be added or removed from the update-refs + * file based on the new todo list. + */ + todo_list_filter_update_refs(r, new_todo); + return 0; } diff --git a/sequencer.c b/sequencer.c index 2d89b3b727a..2808c027d68 100644 --- a/sequencer.c +++ b/sequencer.c @@ -4141,6 +4141,102 @@ cleanup: return result; } +/* + * Parse the update-refs file for the current rebase, then remove the + * refs that do not appear in the todo_list (and have not had updated + * values stored) and add refs that are in the todo_list but not + * represented in the update-refs file. + * + * If there are changes to the update-refs list, then write the new state + * to disk. + */ +void todo_list_filter_update_refs(struct repository *r, + struct todo_list *todo_list) +{ + int i; + int updated = 0; + struct string_list update_refs = STRING_LIST_INIT_DUP; + + sequencer_get_update_refs_state(r->gitdir, &update_refs); + + /* + * For each item in the update_refs list, if it has no updated + * value and does not appear in the todo_list, then remove it + * from the update_refs list. + */ + for (i = 0; i < update_refs.nr; i++) { + int j; + int found = 0; + const char *ref = update_refs.items[i].string; + size_t reflen = strlen(ref); + struct update_ref_record *rec = update_refs.items[i].util; + + /* OID already stored as updated. */ + if (!is_null_oid(&rec->after)) + continue; + + for (j = 0; !found && j < todo_list->total_nr; j++) { + struct todo_item *item = &todo_list->items[j]; + const char *arg = todo_list->buf.buf + item->arg_offset; + + if (item->command != TODO_UPDATE_REF) + continue; + + if (item->arg_len != reflen || + strncmp(arg, ref, reflen)) + continue; + + found = 1; + } + + if (!found) { + free(update_refs.items[i].string); + free(update_refs.items[i].util); + + update_refs.nr--; + MOVE_ARRAY(update_refs.items + i, update_refs.items + i + 1, update_refs.nr - i); + + updated = 1; + i--; + } + } + + /* + * For each todo_item, check if its ref is in the update_refs list. + * If not, then add it as an un-updated ref. + */ + for (i = 0; i < todo_list->total_nr; i++) { + struct todo_item *item = &todo_list->items[i]; + const char *arg = todo_list->buf.buf + item->arg_offset; + int j, found = 0; + + if (item->command != TODO_UPDATE_REF) + continue; + + for (j = 0; !found && j < update_refs.nr; j++) { + const char *ref = update_refs.items[j].string; + + found = strlen(ref) == item->arg_len && + !strncmp(ref, arg, item->arg_len); + } + + if (!found) { + struct string_list_item *inserted; + struct strbuf argref = STRBUF_INIT; + + strbuf_add(&argref, arg, item->arg_len); + inserted = string_list_insert(&update_refs, argref.buf); + inserted->util = init_update_ref_record(argref.buf); + strbuf_release(&argref); + updated = 1; + } + } + + if (updated) + write_update_refs_state(&update_refs); + string_list_clear(&update_refs, 1); +} + static int do_update_ref(struct repository *r, const char *refname) { struct string_list_item *item; diff --git a/sequencer.h b/sequencer.h index e671d7e0743..98f3bcc1658 100644 --- a/sequencer.h +++ b/sequencer.h @@ -132,6 +132,18 @@ void todo_list_release(struct todo_list *todo_list); const char *todo_item_get_arg(struct todo_list *todo_list, struct todo_item *item); +/* + * Parse the update-refs file for the current rebase, then remove the + * refs that do not appear in the todo_list (and have not had updated + * values stored) and add refs that are in the todo_list but not + * represented in the update-refs file. + * + * If there are changes to the update-refs list, then write the new state + * to disk. + */ +void todo_list_filter_update_refs(struct repository *r, + struct todo_list *todo_list); + /* Call this to setup defaults before parsing command line options */ void sequencer_init_config(struct replay_opts *opts); int sequencer_pick_revisions(struct repository *repo, diff --git a/t/lib-rebase.sh b/t/lib-rebase.sh index ec6b9b107da..b57541356bd 100644 --- a/t/lib-rebase.sh +++ b/t/lib-rebase.sh @@ -207,3 +207,18 @@ check_reworded_commits () { >reword-log && test_cmp reword-expected reword-log } + +# usage: set_replace_editor <file> +# +# Replace the todo file with the exact contents of the given file. +set_replace_editor () { + cat >script <<-\EOF && + cat FILENAME >"$1" + + echo 'rebase -i script after editing:' + cat "$1" + EOF + + sed -e "s/FILENAME/$1/g" <script | write_script fake-editor.sh && + test_set_editor "$(pwd)/fake-editor.sh" +} diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index a37820fa728..7bfbd405ab8 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1830,6 +1830,145 @@ test_expect_success '--update-refs updates refs correctly' ' test_cmp_rev HEAD refs/heads/no-conflict-branch ' +test_expect_success 'respect user edits to update-ref steps' ' + git checkout -B update-refs-break no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git branch -f unseen base && + + # First, we will add breaks to the expected todo file + cat >fake-todo-1 <<-EOF && + pick $(git rev-parse HEAD~3) + break + update-ref refs/heads/second + update-ref refs/heads/first + + pick $(git rev-parse HEAD~2) + pick $(git rev-parse HEAD~1) + update-ref refs/heads/third + + pick $(git rev-parse HEAD) + update-ref refs/heads/no-conflict-branch + EOF + + # Second, we will drop some update-refs commands (and move one) + cat >fake-todo-2 <<-EOF && + update-ref refs/heads/second + + pick $(git rev-parse HEAD~2) + update-ref refs/heads/third + pick $(git rev-parse HEAD~1) + break + + pick $(git rev-parse HEAD) + EOF + + # Third, we will: + # * insert a new one (new-branch), + # * re-add an old one (first), and + # * add a second instance of a previously-stored one (second) + cat >fake-todo-3 <<-EOF && + update-ref refs/heads/unseen + update-ref refs/heads/new-branch + pick $(git rev-parse HEAD) + update-ref refs/heads/first + update-ref refs/heads/second + EOF + + ( + set_replace_editor fake-todo-1 && + git rebase -i --update-refs primary && + + # These branches are currently locked. + for b in first second third no-conflict-branch + do + test_must_fail git branch -f $b base || return 1 + done && + + set_replace_editor fake-todo-2 && + git rebase --edit-todo && + + # These branches are currently locked. + for b in second third + do + test_must_fail git branch -f $b base || return 1 + done && + + # These branches are currently unlocked for checkout. + for b in first no-conflict-branch + do + git worktree add wt-$b $b && + git worktree remove wt-$b || return 1 + done && + + git rebase --continue && + + set_replace_editor fake-todo-3 && + git rebase --edit-todo && + + # These branches are currently locked. + for b in second third first unseen + do + test_must_fail git branch -f $b base || return 1 + done && + + # These branches are currently unlocked for checkout. + for b in no-conflict-branch + do + git worktree add wt-$b $b && + git worktree remove wt-$b || return 1 + done && + + git rebase --continue + ) && + + test_cmp_rev HEAD~2 refs/heads/third && + test_cmp_rev HEAD~1 refs/heads/unseen && + test_cmp_rev HEAD~1 refs/heads/new-branch && + test_cmp_rev HEAD refs/heads/first && + test_cmp_rev HEAD refs/heads/second && + test_cmp_rev HEAD refs/heads/no-conflict-branch +' + +test_expect_success '--update-refs: check failed ref update' ' + git checkout -B update-refs-error no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~2 && + git branch -f third HEAD~1 && + + cat >fake-todo <<-EOF && + pick $(git rev-parse HEAD~3) + break + update-ref refs/heads/first + + pick $(git rev-parse HEAD~2) + update-ref refs/heads/second + + pick $(git rev-parse HEAD~1) + update-ref refs/heads/third + + pick $(git rev-parse HEAD) + update-ref refs/heads/no-conflict-branch + EOF + + ( + set_replace_editor fake-todo && + git rebase -i --update-refs base + ) && + + # At this point, the values of first, second, and third are + # recorded in the update-refs file. We will force-update the + # "second" ref, but "git branch -f" will not work because of + # the lock in the update-refs file. + git rev-parse third >.git/refs/heads/second && + + git rebase --continue 2>err && + grep "update_ref failed for ref '\''refs/heads/second'\''" err +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v4 09/12] sequencer: rewrite update-refs as user edits todo list 2022-07-12 13:07 ` [PATCH v4 09/12] sequencer: rewrite update-refs as user edits todo list Derrick Stolee via GitGitGadget @ 2022-07-15 10:27 ` Phillip Wood 2022-07-15 13:13 ` Derrick Stolee 2022-07-16 19:20 ` Elijah Newren 1 sibling, 1 reply; 144+ messages in thread From: Phillip Wood @ 2022-07-15 10:27 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee Hi Stolee On 12/07/2022 14:07, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > An interactive rebase provides opportunities for the user to edit the > todo list. The --update-refs option initializes the list with some > 'update-ref <ref>' steps, but the user could add these manually. > Further, the user could add or remove these steps during pauses in the > interactive rebase. > > Add a new method, todo_list_filter_update_refs(), that scans a todo_list > and compares it to the stored update-refs file. There are two actions > that can happen at this point: > > 1. If a '<ref>/<before>/<after>' triple in the update-refs file does not > have a matching 'update-ref <ref>' command in the todo-list _and_ the > <after> value is the null OID, then remove that triple. Here, the > user removed the 'update-ref <ref>' command before it was executed, > since if it was executed then the <after> value would store the > commit at that position. > > 2. If a 'update-ref <ref>' command in the todo-list does not have a > matching '<ref>/<before>/<after>' triple in the update-refs file, > then insert a new one. Store the <before> value to be the current > OID pointed at by <ref>. This is handled inside of the > init_update_ref_record() helper method. > > We can test that this works by rewriting the todo-list several times in > the course of a rebase. Check that each ref is locked or unlocked for > updates after each todo-list update. We an also verify that the ref > update fails if a concurrent process updates one of the refs after the > rebase process records the "locked" ref location. Thanks for adding this and taking the time to write some good tests. When adding a new update-ref command to the todo list it would be nice to check if the ref is already checked. We could print a warning or force the user to re-edit the todo list as we do if they delete a pick and rebase.missingcommitscheck == error The checks below are quadratic but we don't expect that many refs so that should be fine. I don't think it is worth the complexity of building a hash table or whatever at this stage. Best Wishes Phillip > To help these tests, add a new 'set_replace_editor' helper that will > replace the todo-list with an exact file. > > Reported-by: Phillip Wood <phillip.wood123@gmail.com> > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > rebase-interactive.c | 6 ++ > sequencer.c | 96 +++++++++++++++++++++++ > sequencer.h | 12 +++ > t/lib-rebase.sh | 15 ++++ > t/t3404-rebase-interactive.sh | 139 ++++++++++++++++++++++++++++++++++ > 5 files changed, 268 insertions(+) > > diff --git a/rebase-interactive.c b/rebase-interactive.c > index 1ff07647af3..7407c593191 100644 > --- a/rebase-interactive.c > +++ b/rebase-interactive.c > @@ -146,6 +146,12 @@ int edit_todo_list(struct repository *r, struct todo_list *todo_list, > return -4; > } > > + /* > + * See if branches need to be added or removed from the update-refs > + * file based on the new todo list. > + */ > + todo_list_filter_update_refs(r, new_todo); > + > return 0; > } > > diff --git a/sequencer.c b/sequencer.c > index 2d89b3b727a..2808c027d68 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -4141,6 +4141,102 @@ cleanup: > return result; > } > > +/* > + * Parse the update-refs file for the current rebase, then remove the > + * refs that do not appear in the todo_list (and have not had updated > + * values stored) and add refs that are in the todo_list but not > + * represented in the update-refs file. > + * > + * If there are changes to the update-refs list, then write the new state > + * to disk. > + */ > +void todo_list_filter_update_refs(struct repository *r, > + struct todo_list *todo_list) > +{ > + int i; > + int updated = 0; > + struct string_list update_refs = STRING_LIST_INIT_DUP; > + > + sequencer_get_update_refs_state(r->gitdir, &update_refs); > + > + /* > + * For each item in the update_refs list, if it has no updated > + * value and does not appear in the todo_list, then remove it > + * from the update_refs list. > + */ > + for (i = 0; i < update_refs.nr; i++) { > + int j; > + int found = 0; > + const char *ref = update_refs.items[i].string; > + size_t reflen = strlen(ref); > + struct update_ref_record *rec = update_refs.items[i].util; > + > + /* OID already stored as updated. */ > + if (!is_null_oid(&rec->after)) > + continue; > + > + for (j = 0; !found && j < todo_list->total_nr; j++) { > + struct todo_item *item = &todo_list->items[j]; > + const char *arg = todo_list->buf.buf + item->arg_offset; > + > + if (item->command != TODO_UPDATE_REF) > + continue; > + > + if (item->arg_len != reflen || > + strncmp(arg, ref, reflen)) > + continue; > + > + found = 1; > + } > + > + if (!found) { > + free(update_refs.items[i].string); > + free(update_refs.items[i].util); > + > + update_refs.nr--; > + MOVE_ARRAY(update_refs.items + i, update_refs.items + i + 1, update_refs.nr - i); > + > + updated = 1; > + i--; > + } > + } > + > + /* > + * For each todo_item, check if its ref is in the update_refs list. > + * If not, then add it as an un-updated ref. > + */ > + for (i = 0; i < todo_list->total_nr; i++) { > + struct todo_item *item = &todo_list->items[i]; > + const char *arg = todo_list->buf.buf + item->arg_offset; > + int j, found = 0; > + > + if (item->command != TODO_UPDATE_REF) > + continue; > + > + for (j = 0; !found && j < update_refs.nr; j++) { > + const char *ref = update_refs.items[j].string; > + > + found = strlen(ref) == item->arg_len && > + !strncmp(ref, arg, item->arg_len); > + } > + > + if (!found) { > + struct string_list_item *inserted; > + struct strbuf argref = STRBUF_INIT; > + > + strbuf_add(&argref, arg, item->arg_len); > + inserted = string_list_insert(&update_refs, argref.buf); > + inserted->util = init_update_ref_record(argref.buf); > + strbuf_release(&argref); > + updated = 1; > + } > + } > + > + if (updated) > + write_update_refs_state(&update_refs); > + string_list_clear(&update_refs, 1); > +} > + > static int do_update_ref(struct repository *r, const char *refname) > { > struct string_list_item *item; > diff --git a/sequencer.h b/sequencer.h > index e671d7e0743..98f3bcc1658 100644 > --- a/sequencer.h > +++ b/sequencer.h > @@ -132,6 +132,18 @@ void todo_list_release(struct todo_list *todo_list); > const char *todo_item_get_arg(struct todo_list *todo_list, > struct todo_item *item); > > +/* > + * Parse the update-refs file for the current rebase, then remove the > + * refs that do not appear in the todo_list (and have not had updated > + * values stored) and add refs that are in the todo_list but not > + * represented in the update-refs file. > + * > + * If there are changes to the update-refs list, then write the new state > + * to disk. > + */ > +void todo_list_filter_update_refs(struct repository *r, > + struct todo_list *todo_list); > + > /* Call this to setup defaults before parsing command line options */ > void sequencer_init_config(struct replay_opts *opts); > int sequencer_pick_revisions(struct repository *repo, > diff --git a/t/lib-rebase.sh b/t/lib-rebase.sh > index ec6b9b107da..b57541356bd 100644 > --- a/t/lib-rebase.sh > +++ b/t/lib-rebase.sh > @@ -207,3 +207,18 @@ check_reworded_commits () { > >reword-log && > test_cmp reword-expected reword-log > } > + > +# usage: set_replace_editor <file> > +# > +# Replace the todo file with the exact contents of the given file. > +set_replace_editor () { > + cat >script <<-\EOF && > + cat FILENAME >"$1" > + > + echo 'rebase -i script after editing:' > + cat "$1" > + EOF > + > + sed -e "s/FILENAME/$1/g" <script | write_script fake-editor.sh && > + test_set_editor "$(pwd)/fake-editor.sh" > +} > diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh > index a37820fa728..7bfbd405ab8 100755 > --- a/t/t3404-rebase-interactive.sh > +++ b/t/t3404-rebase-interactive.sh > @@ -1830,6 +1830,145 @@ test_expect_success '--update-refs updates refs correctly' ' > test_cmp_rev HEAD refs/heads/no-conflict-branch > ' > > +test_expect_success 'respect user edits to update-ref steps' ' > + git checkout -B update-refs-break no-conflict-branch && > + git branch -f base HEAD~4 && > + git branch -f first HEAD~3 && > + git branch -f second HEAD~3 && > + git branch -f third HEAD~1 && > + git branch -f unseen base && > + > + # First, we will add breaks to the expected todo file > + cat >fake-todo-1 <<-EOF && > + pick $(git rev-parse HEAD~3) > + break > + update-ref refs/heads/second > + update-ref refs/heads/first > + > + pick $(git rev-parse HEAD~2) > + pick $(git rev-parse HEAD~1) > + update-ref refs/heads/third > + > + pick $(git rev-parse HEAD) > + update-ref refs/heads/no-conflict-branch > + EOF > + > + # Second, we will drop some update-refs commands (and move one) > + cat >fake-todo-2 <<-EOF && > + update-ref refs/heads/second > + > + pick $(git rev-parse HEAD~2) > + update-ref refs/heads/third > + pick $(git rev-parse HEAD~1) > + break > + > + pick $(git rev-parse HEAD) > + EOF > + > + # Third, we will: > + # * insert a new one (new-branch), > + # * re-add an old one (first), and > + # * add a second instance of a previously-stored one (second) > + cat >fake-todo-3 <<-EOF && > + update-ref refs/heads/unseen > + update-ref refs/heads/new-branch > + pick $(git rev-parse HEAD) > + update-ref refs/heads/first > + update-ref refs/heads/second > + EOF > + > + ( > + set_replace_editor fake-todo-1 && > + git rebase -i --update-refs primary && > + > + # These branches are currently locked. > + for b in first second third no-conflict-branch > + do > + test_must_fail git branch -f $b base || return 1 > + done && > + > + set_replace_editor fake-todo-2 && > + git rebase --edit-todo && > + > + # These branches are currently locked. > + for b in second third > + do > + test_must_fail git branch -f $b base || return 1 > + done && > + > + # These branches are currently unlocked for checkout. > + for b in first no-conflict-branch > + do > + git worktree add wt-$b $b && > + git worktree remove wt-$b || return 1 > + done && > + > + git rebase --continue && > + > + set_replace_editor fake-todo-3 && > + git rebase --edit-todo && > + > + # These branches are currently locked. > + for b in second third first unseen > + do > + test_must_fail git branch -f $b base || return 1 > + done && > + > + # These branches are currently unlocked for checkout. > + for b in no-conflict-branch > + do > + git worktree add wt-$b $b && > + git worktree remove wt-$b || return 1 > + done && > + > + git rebase --continue > + ) && > + > + test_cmp_rev HEAD~2 refs/heads/third && > + test_cmp_rev HEAD~1 refs/heads/unseen && > + test_cmp_rev HEAD~1 refs/heads/new-branch && > + test_cmp_rev HEAD refs/heads/first && > + test_cmp_rev HEAD refs/heads/second && > + test_cmp_rev HEAD refs/heads/no-conflict-branch > +' > + > +test_expect_success '--update-refs: check failed ref update' ' > + git checkout -B update-refs-error no-conflict-branch && > + git branch -f base HEAD~4 && > + git branch -f first HEAD~3 && > + git branch -f second HEAD~2 && > + git branch -f third HEAD~1 && > + > + cat >fake-todo <<-EOF && > + pick $(git rev-parse HEAD~3) > + break > + update-ref refs/heads/first > + > + pick $(git rev-parse HEAD~2) > + update-ref refs/heads/second > + > + pick $(git rev-parse HEAD~1) > + update-ref refs/heads/third > + > + pick $(git rev-parse HEAD) > + update-ref refs/heads/no-conflict-branch > + EOF > + > + ( > + set_replace_editor fake-todo && > + git rebase -i --update-refs base > + ) && > + > + # At this point, the values of first, second, and third are > + # recorded in the update-refs file. We will force-update the > + # "second" ref, but "git branch -f" will not work because of > + # the lock in the update-refs file. > + git rev-parse third >.git/refs/heads/second && > + > + git rebase --continue 2>err && > + grep "update_ref failed for ref '\''refs/heads/second'\''" err > +' > + > # This must be the last test in this file > test_expect_success '$EDITOR and friends are unchanged' ' > test_editor_unchanged ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 09/12] sequencer: rewrite update-refs as user edits todo list 2022-07-15 10:27 ` Phillip Wood @ 2022-07-15 13:13 ` Derrick Stolee 2022-07-18 13:09 ` Phillip Wood 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-07-15 13:13 UTC (permalink / raw) To: phillip.wood, Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren On 7/15/2022 6:27 AM, Phillip Wood wrote: >> We can test that this works by rewriting the todo-list several times in >> the course of a rebase. Check that each ref is locked or unlocked for >> updates after each todo-list update. We an also verify that the ref >> update fails if a concurrent process updates one of the refs after the >> rebase process records the "locked" ref location. > > Thanks for adding this and taking the time to write some good tests. When adding a new update-ref command to the todo list it would be nice to check if the ref is already checked. We could print a warning or force the user to re-edit the todo list as we do if they delete a pick and rebase.missingcommitscheck == error I agree that that would be a helpful feature instead of the user discovering the issue at the end of the rebase (as they do with the message from patch 12). > The checks below are quadratic but we don't expect that many refs so that should be fine. I don't think it is worth the complexity of building a hash table or whatever at this stage. A hash table would be the natural approach if we see users having trouble from such high use of the feature. I think both of these concerns would be excellent for a follow-up, since they would shave off some rough edges. I hesitate to add them to this series since it has been growing quite a bit already. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 09/12] sequencer: rewrite update-refs as user edits todo list 2022-07-15 13:13 ` Derrick Stolee @ 2022-07-18 13:09 ` Phillip Wood 0 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-07-18 13:09 UTC (permalink / raw) To: Derrick Stolee, phillip.wood, Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren Hi Stolee On 15/07/2022 14:13, Derrick Stolee wrote: > On 7/15/2022 6:27 AM, Phillip Wood wrote: > [...] > I think both of these concerns would be excellent for a follow-up, > since they would shave off some rough edges. I hesitate to add them > to this series since it has been growing quite a bit already. Yes that makes sense (and applies to pretty much all of my other suggestions as well). This series is looking good and it makes sense to get it merged and hopefully get some user feedback before making any tweaks. I'm going to off the list most of the next three weeks, so I probably wont reply promptly to any reroll. I think that what you've got in this version is pretty much ready so I'd be very happy to see this topic merged by the time I get back on the list. Best Wishes Phillip > Thanks, > -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 09/12] sequencer: rewrite update-refs as user edits todo list 2022-07-12 13:07 ` [PATCH v4 09/12] sequencer: rewrite update-refs as user edits todo list Derrick Stolee via GitGitGadget 2022-07-15 10:27 ` Phillip Wood @ 2022-07-16 19:20 ` Elijah Newren 1 sibling, 0 replies; 144+ messages in thread From: Elijah Newren @ 2022-07-16 19:20 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood, Derrick Stolee On Tue, Jul 12, 2022 at 6:07 AM Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com> wrote: > > From: Derrick Stolee <derrickstolee@github.com> > > An interactive rebase provides opportunities for the user to edit the > todo list. The --update-refs option initializes the list with some > 'update-ref <ref>' steps, but the user could add these manually. > Further, the user could add or remove these steps during pauses in the > interactive rebase. > > Add a new method, todo_list_filter_update_refs(), that scans a todo_list > and compares it to the stored update-refs file. There are two actions > that can happen at this point: > > 1. If a '<ref>/<before>/<after>' triple in the update-refs file does not > have a matching 'update-ref <ref>' command in the todo-list _and_ the > <after> value is the null OID, then remove that triple. Here, the > user removed the 'update-ref <ref>' command before it was executed, > since if it was executed then the <after> value would store the > commit at that position. > > 2. If a 'update-ref <ref>' command in the todo-list does not have a > matching '<ref>/<before>/<after>' triple in the update-refs file, > then insert a new one. Store the <before> value to be the current > OID pointed at by <ref>. This is handled inside of the > init_update_ref_record() helper method. > > We can test that this works by rewriting the todo-list several times in > the course of a rebase. Check that each ref is locked or unlocked for > updates after each todo-list update. We an also verify that the ref s/an/can/ ? ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v4 10/12] rebase: add rebase.updateRefs config option 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (8 preceding siblings ...) 2022-07-12 13:07 ` [PATCH v4 09/12] sequencer: rewrite update-refs as user edits todo list Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 ` Derrick Stolee via GitGitGadget 2022-07-12 13:07 ` [PATCH v4 11/12] sequencer: ignore HEAD ref under --update-refs Derrick Stolee via GitGitGadget ` (4 subsequent siblings) 14 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The previous change added the --update-refs command-line option. For users who always want this mode, create the rebase.updateRefs config option which behaves the same way as rebase.autoSquash does with the --autosquash option. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- Documentation/config/rebase.txt | 3 +++ Documentation/git-rebase.txt | 3 +++ builtin/rebase.c | 5 +++++ t/t3404-rebase-interactive.sh | 14 ++++++++++++++ 4 files changed, 25 insertions(+) diff --git a/Documentation/config/rebase.txt b/Documentation/config/rebase.txt index 8c979cb20f2..f19bd0e0407 100644 --- a/Documentation/config/rebase.txt +++ b/Documentation/config/rebase.txt @@ -21,6 +21,9 @@ rebase.autoStash:: `--autostash` options of linkgit:git-rebase[1]. Defaults to false. +rebase.updateRefs:: + If set to true enable `--update-refs` option by default. + rebase.missingCommitsCheck:: If set to "warn", git rebase -i will print a warning if some commits are removed (e.g. a line was deleted), however the diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index e7611b4089c..1249f9ed617 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -615,6 +615,9 @@ start would be overridden by the presence of are being rebased. Any branches that are checked out in a worktree or point to a `squash! ...` or `fixup! ...` commit are not updated in this way. ++ +If the configuration variable `rebase.updateRefs` is set, then this option +can be used to override and disable this setting. INCOMPATIBLE OPTIONS -------------------- diff --git a/builtin/rebase.c b/builtin/rebase.c index 56d82a52106..8ebc98ea505 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -802,6 +802,11 @@ static int rebase_config(const char *var, const char *value, void *data) return 0; } + if (!strcmp(var, "rebase.updaterefs")) { + opts->update_refs = git_config_bool(var, value); + return 0; + } + if (!strcmp(var, "rebase.reschedulefailedexec")) { opts->reschedule_failed_exec = git_config_bool(var, value); return 0; diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 7bfbd405ab8..4b7b77a4123 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1768,6 +1768,12 @@ test_expect_success '--update-refs adds label and update-ref commands' ' EOF test_must_fail git rebase -i --autosquash --update-refs primary >todo && + test_cmp expect todo && + + test_must_fail git -c rebase.autosquash=true \ + -c rebase.updaterefs=true \ + rebase -i primary >todo && + test_cmp expect todo ) ' @@ -1809,6 +1815,14 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' --rebase-merges=rebase-cousins \ --update-refs primary >todo && + test_cmp expect todo && + + test_must_fail git -c rebase.autosquash=true \ + -c rebase.updaterefs=true \ + rebase -i \ + --rebase-merges=rebase-cousins \ + primary >todo && + test_cmp expect todo ) ' -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v4 11/12] sequencer: ignore HEAD ref under --update-refs 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (9 preceding siblings ...) 2022-07-12 13:07 ` [PATCH v4 10/12] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 ` Derrick Stolee via GitGitGadget 2022-07-12 13:07 ` [PATCH v4 12/12] sequencer: notify user of --update-refs activity Derrick Stolee via GitGitGadget ` (3 subsequent siblings) 14 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> When using the 'git rebase -i --update-refs' option, the todo list is populated with 'update-ref' commands for all tip refs in the history that is being rebased. Refs that are checked out by some worktree are instead added as a comment to warn the user that they will not be updated. Until now, this included the HEAD ref, which is being updated by the rebase process itself, regardless of the --update-refs option. Remove the comment in this case by ignoring any decorations that match the HEAD ref. Reported-by: Elijah Newren <newren@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/sequencer.c b/sequencer.c index 2808c027d68..82ef062d497 100644 --- a/sequencer.c +++ b/sequencer.c @@ -5855,12 +5855,25 @@ static int add_decorations_to_list(const struct commit *commit, struct todo_add_branch_context *ctx) { const struct name_decoration *decoration = get_name_decoration(&commit->object); + const char *head_ref = resolve_ref_unsafe("HEAD", + RESOLVE_REF_READING, + NULL, + NULL); while (decoration) { struct todo_item *item; const char *path; size_t base_offset = ctx->buf->len; + /* + * If the branch is the current HEAD, then it will be + * updated by the default rebase behavior. + */ + if (head_ref && !strcmp(head_ref, decoration->name)) { + decoration = decoration->next; + continue; + } + ALLOC_GROW(ctx->items, ctx->items_nr + 1, ctx->items_alloc); -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v4 12/12] sequencer: notify user of --update-refs activity 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (10 preceding siblings ...) 2022-07-12 13:07 ` [PATCH v4 11/12] sequencer: ignore HEAD ref under --update-refs Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 ` Derrick Stolee via GitGitGadget 2022-07-15 10:12 ` Phillip Wood 2022-07-16 22:09 ` Elijah Newren 2022-07-12 15:37 ` [PATCH v4 00/12] rebase: update branches in multi-part topic Junio C Hamano ` (2 subsequent siblings) 14 siblings, 2 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-12 13:07 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> When the user runs 'git rebase -i --update-refs', the end message still says only Successfully rebased and updated <HEAD-ref>. Update the sequencer to collect the successful (and unsuccessful) ref updates due to the --update-refs option, so the end message now says Successfully rebased and updated <HEAD-ref>. Updated the following refs with --update-refs: refs/heads/first refs/heads/third Failed to update the following refs with --update-refs: refs/heads/second To test this output, we need to be very careful to format the expected error to drop the leading tab characters. Also, we need to be aware that the verbose output from 'git rebase' is writing progress lines which don't use traditional newlines but clear the line after every progress item is complete. When opening the error file in an editor, these lines are visible, but when looking at the diff in a terminal those lines disappear because of the characters that delete the previous characters. Use 'sed' to clear those progress lines and clear the tabs so we can get an exact match on our expected output. Reported-by: Elijah Newren <newren@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 40 +++++++++++++++++++++++++++++------ t/t3404-rebase-interactive.sh | 35 +++++++++++++++++++++++++++--- 2 files changed, 66 insertions(+), 9 deletions(-) diff --git a/sequencer.c b/sequencer.c index 82ef062d497..bdc67c66f3e 100644 --- a/sequencer.c +++ b/sequencer.c @@ -4257,17 +4257,20 @@ static int do_update_ref(struct repository *r, const char *refname) return 0; } -static int do_update_refs(struct repository *r) +static int do_update_refs(struct repository *r, int quiet) { int res = 0; struct string_list_item *item; struct string_list refs_to_oids = STRING_LIST_INIT_DUP; struct ref_store *refs = get_main_ref_store(r); + struct strbuf update_msg = STRBUF_INIT; + struct strbuf error_msg = STRBUF_INIT; sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); for_each_string_list_item(item, &refs_to_oids) { struct update_ref_record *rec = item->util; + int loop_res; if (oideq(&rec->after, the_hash_algo->null_oid)) { /* @@ -4277,13 +4280,38 @@ static int do_update_refs(struct repository *r) continue; } - res |= refs_update_ref(refs, "rewritten during rebase", - item->string, - &rec->after, &rec->before, - 0, UPDATE_REFS_MSG_ON_ERR); + loop_res = refs_update_ref(refs, "rewritten during rebase", + item->string, + &rec->after, &rec->before, + 0, UPDATE_REFS_MSG_ON_ERR); + res |= loop_res; + + if (quiet) + continue; + + if (loop_res) + strbuf_addf(&error_msg, "\t%s\n", item->string); + else + strbuf_addf(&update_msg, "\t%s\n", item->string); + } + + if (!quiet && + (update_msg.len || error_msg.len)) { + fprintf(stderr, + _("Updated the following refs with %s:\n%s"), + "--update-refs", + update_msg.buf); + + if (res) + fprintf(stderr, + _("Failed to update the following refs with %s:\n%s"), + "--update-refs", + error_msg.buf); } string_list_clear(&refs_to_oids, 1); + strbuf_release(&update_msg); + strbuf_release(&error_msg); return res; } @@ -4804,7 +4832,7 @@ cleanup_head_ref: strbuf_release(&head_ref); } - do_update_refs(r); + do_update_refs(r, opts->quiet); /* * Sequence of picks finished successfully; cleanup by diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 4b7b77a4123..ef902b5431f 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1836,12 +1836,26 @@ test_expect_success '--update-refs updates refs correctly' ' test_commit extra2 fileX && git commit --amend --fixup=L && - git rebase -i --autosquash --update-refs primary && + git rebase -i --autosquash --update-refs primary 2>err && test_cmp_rev HEAD~3 refs/heads/first && test_cmp_rev HEAD~3 refs/heads/second && test_cmp_rev HEAD~1 refs/heads/third && - test_cmp_rev HEAD refs/heads/no-conflict-branch + test_cmp_rev HEAD refs/heads/no-conflict-branch && + + cat >expect <<-\EOF && + Successfully rebased and updated refs/heads/update-refs. + Updated the following refs with --update-refs: + refs/heads/first + refs/heads/no-conflict-branch + refs/heads/second + refs/heads/third + EOF + + # Clear "Rebasing (X/Y)" progress lines and drop leading tabs. + sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \ + <err >err.trimmed && + test_cmp expect err.trimmed ' test_expect_success 'respect user edits to update-ref steps' ' @@ -1980,7 +1994,22 @@ test_expect_success '--update-refs: check failed ref update' ' git rev-parse third >.git/refs/heads/second && git rebase --continue 2>err && - grep "update_ref failed for ref '\''refs/heads/second'\''" err + grep "update_ref failed for ref '\''refs/heads/second'\''" err && + + cat >expect <<-\EOF && + Updated the following refs with --update-refs: + refs/heads/first + refs/heads/no-conflict-branch + refs/heads/third + Failed to update the following refs with --update-refs: + refs/heads/second + EOF + + # Clear "Rebasing (X/Y)" progress lines and drop leading tabs. + tail -n 6 err >err.last && + sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \ + <err.last >err.trimmed && + test_cmp expect err.trimmed ' # This must be the last test in this file -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v4 12/12] sequencer: notify user of --update-refs activity 2022-07-12 13:07 ` [PATCH v4 12/12] sequencer: notify user of --update-refs activity Derrick Stolee via GitGitGadget @ 2022-07-15 10:12 ` Phillip Wood 2022-07-15 13:20 ` Derrick Stolee 2022-07-16 22:09 ` Elijah Newren 1 sibling, 1 reply; 144+ messages in thread From: Phillip Wood @ 2022-07-15 10:12 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee On 12/07/2022 14:07, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > When the user runs 'git rebase -i --update-refs', the end message still > says only > > Successfully rebased and updated <HEAD-ref>. > > Update the sequencer to collect the successful (and unsuccessful) ref > updates due to the --update-refs option, so the end message now says > > Successfully rebased and updated <HEAD-ref>. > Updated the following refs with --update-refs: > refs/heads/first > refs/heads/third > Failed to update the following refs with --update-refs: > refs/heads/second > > To test this output, we need to be very careful to format the expected > error to drop the leading tab characters. Also, we need to be aware that > the verbose output from 'git rebase' is writing progress lines which s/is writing/writes/ ? > don't use traditional newlines but clear the line after every progress > item is complete. I was a bit confused by the reference to "verbose output" in this paragraph. When the user passes --verbose then we do actually use NL, it is when the user does not pass verbose that we use CR instead. > When opening the error file in an editor, these lines > are visible, but when looking at the diff in a terminal those lines > disappear because of the characters that delete the previous characters. > Use 'sed' to clear those progress lines and clear the tabs so we can get > an exact match on our expected output. Thanks for the comprehensive commit message and for implementing an excellent suggestion from Elijah. I wonder if it makes sense to distinguish between the current branch and all the others when writing the update message as we do here or if all the refs should just be in a single list. I also think it doesn't matter much and we can change it later if we want. From the last test it looks like we are already printing something when we fail to update a ref (possibly this comes from the refs backend code) I don't think it hurts to print a summary of them all after that though. > Reported-by: Elijah Newren <newren@gmail.com> > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > --- > sequencer.c | 40 +++++++++++++++++++++++++++++------ > t/t3404-rebase-interactive.sh | 35 +++++++++++++++++++++++++++--- > 2 files changed, 66 insertions(+), 9 deletions(-) > > diff --git a/sequencer.c b/sequencer.c > index 82ef062d497..bdc67c66f3e 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -4257,17 +4257,20 @@ static int do_update_ref(struct repository *r, const char *refname) > return 0; > } > > -static int do_update_refs(struct repository *r) > +static int do_update_refs(struct repository *r, int quiet) > { > int res = 0; > struct string_list_item *item; > struct string_list refs_to_oids = STRING_LIST_INIT_DUP; > struct ref_store *refs = get_main_ref_store(r); > + struct strbuf update_msg = STRBUF_INIT; > + struct strbuf error_msg = STRBUF_INIT; > > sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); > > for_each_string_list_item(item, &refs_to_oids) { > struct update_ref_record *rec = item->util; > + int loop_res; > > if (oideq(&rec->after, the_hash_algo->null_oid)) { > /* > @@ -4277,13 +4280,38 @@ static int do_update_refs(struct repository *r) > continue; > } > > - res |= refs_update_ref(refs, "rewritten during rebase", > - item->string, > - &rec->after, &rec->before, > - 0, UPDATE_REFS_MSG_ON_ERR); > + loop_res = refs_update_ref(refs, "rewritten during rebase", > + item->string, > + &rec->after, &rec->before, > + 0, UPDATE_REFS_MSG_ON_ERR); > + res |= loop_res; > + > + if (quiet) > + continue; > + > + if (loop_res) > + strbuf_addf(&error_msg, "\t%s\n", item->string); > + else > + strbuf_addf(&update_msg, "\t%s\n", item->string); > + } > + > + if (!quiet && As you skip adding anything to the strbufs when quiet is true you don't really need this test > + (update_msg.len || error_msg.len)) { > + fprintf(stderr, > + _("Updated the following refs with %s:\n%s"), > + "--update-refs", > + update_msg.buf); This will be printed even if all the updates falied > + > + if (res) > + fprintf(stderr, > + _("Failed to update the following refs with %s:\n%s"), > + "--update-refs", > + error_msg.buf); > } > > string_list_clear(&refs_to_oids, 1); > + strbuf_release(&update_msg); > + strbuf_release(&error_msg); > return res; > } > > @@ -4804,7 +4832,7 @@ cleanup_head_ref: > strbuf_release(&head_ref); > } > > - do_update_refs(r); > + do_update_refs(r, opts->quiet); > > /* > * Sequence of picks finished successfully; cleanup by > diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh > index 4b7b77a4123..ef902b5431f 100755 > --- a/t/t3404-rebase-interactive.sh > +++ b/t/t3404-rebase-interactive.sh > @@ -1836,12 +1836,26 @@ test_expect_success '--update-refs updates refs correctly' ' > test_commit extra2 fileX && > git commit --amend --fixup=L && > > - git rebase -i --autosquash --update-refs primary && > + git rebase -i --autosquash --update-refs primary 2>err && > > test_cmp_rev HEAD~3 refs/heads/first && > test_cmp_rev HEAD~3 refs/heads/second && > test_cmp_rev HEAD~1 refs/heads/third && > - test_cmp_rev HEAD refs/heads/no-conflict-branch > + test_cmp_rev HEAD refs/heads/no-conflict-branch && > + > + cat >expect <<-\EOF && > + Successfully rebased and updated refs/heads/update-refs. > + Updated the following refs with --update-refs: > + refs/heads/first > + refs/heads/no-conflict-branch > + refs/heads/second > + refs/heads/third > + EOF > + > + # Clear "Rebasing (X/Y)" progress lines and drop leading tabs. > + sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \ > + <err >err.trimmed && > + test_cmp expect err.trimmed > ' > > test_expect_success 'respect user edits to update-ref steps' ' > @@ -1980,7 +1994,22 @@ test_expect_success '--update-refs: check failed ref update' ' > git rev-parse third >.git/refs/heads/second && > > git rebase --continue 2>err && > - grep "update_ref failed for ref '\''refs/heads/second'\''" err > + grep "update_ref failed for ref '\''refs/heads/second'\''" err && > + > + cat >expect <<-\EOF && > + Updated the following refs with --update-refs: > + refs/heads/first > + refs/heads/no-conflict-branch > + refs/heads/third > + Failed to update the following refs with --update-refs: > + refs/heads/second > + EOF > + > + # Clear "Rebasing (X/Y)" progress lines and drop leading tabs. > + tail -n 6 err >err.last && I'm curious as to why we need tail here but not in the test above. This is looking good, I'd be happy enough to see it merged as is. Best Wishes Phillip > + sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \ > + <err.last >err.trimmed && > + test_cmp expect err.trimmed > ' > > # This must be the last test in this file ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 12/12] sequencer: notify user of --update-refs activity 2022-07-15 10:12 ` Phillip Wood @ 2022-07-15 13:20 ` Derrick Stolee 2022-07-16 20:51 ` Elijah Newren 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-07-15 13:20 UTC (permalink / raw) To: phillip.wood, Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren On 7/15/2022 6:12 AM, Phillip Wood wrote: > On 12/07/2022 14:07, Derrick Stolee via GitGitGadget wrote: >> From: Derrick Stolee <derrickstolee@github.com> >> >> When the user runs 'git rebase -i --update-refs', the end message still >> says only >> >> Successfully rebased and updated <HEAD-ref>. >> >> Update the sequencer to collect the successful (and unsuccessful) ref >> updates due to the --update-refs option, so the end message now says >> >> Successfully rebased and updated <HEAD-ref>. >> Updated the following refs with --update-refs: >> refs/heads/first >> refs/heads/third >> Failed to update the following refs with --update-refs: >> refs/heads/second >> >> To test this output, we need to be very careful to format the expected >> error to drop the leading tab characters. Also, we need to be aware that >> the verbose output from 'git rebase' is writing progress lines which > > s/is writing/writes/ ? That is an improvement. Thanks. >> don't use traditional newlines but clear the line after every progress >> item is complete. > > I was a bit confused by the reference to "verbose output" in this paragraph. When the user passes --verbose then we do actually use NL, it is when the user does not pass verbose that we use CR instead. I was mostly thinking that the summary message at the end is only shown when the command has verbose output (which really means the default of --no-quiet). There is no current way to have --no-progress --no-quiet to keep the progress lines out of the output but still have the summary. >> When opening the error file in an editor, these lines >> are visible, but when looking at the diff in a terminal those lines >> disappear because of the characters that delete the previous characters. >> Use 'sed' to clear those progress lines and clear the tabs so we can get >> an exact match on our expected output. > > Thanks for the comprehensive commit message and for implementing an excellent suggestion from Elijah. I wonder if it makes sense to distinguish between the current branch and all the others when writing the update message as we do here or if all the refs should just be in a single list. I also think it doesn't matter much and we can change it later if we want. I'm definitely open to suggestions, but I also think we should start somewhere and see what users think. Since the mechanisms for updating the refs are different, I felt it was appropriate to have different error messages. >> + if (!quiet && > > As you skip adding anything to the strbufs when quiet is true you don't really need this test > >> + (update_msg.len || error_msg.len)) { >> + fprintf(stderr, >> + _("Updated the following refs with %s:\n%s"), >> + "--update-refs", >> + update_msg.buf); > > This will be printed even if all the updates falied These are good points! Instead, I should arrange this as if (update_msg.len) /* send the success message. */ if (error_msg.len) /* send the failure message. */ >> + cat >expect <<-\EOF && >> + Updated the following refs with --update-refs: >> + refs/heads/first >> + refs/heads/no-conflict-branch >> + refs/heads/third >> + Failed to update the following refs with --update-refs: >> + refs/heads/second >> + EOF (copying this message from earlier in your message for context) > From the last test it looks like we are already printing something when we fail to update a ref (possibly this comes from the refs backend code) I don't think it hurts to print a summary of them all after that though. >> + # Clear "Rebasing (X/Y)" progress lines and drop leading tabs. >> + tail -n 6 err >err.last && > > I'm curious as to why we need tail here but not in the test above. I think I was trying to avoid testing the error messages from the ref backend code. I bet that since this tail is here, the 'sed' is no longer required. >> + sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \ >> + <err.last >err.trimmed && >> + test_cmp expect err.trimmed Thanks! -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 12/12] sequencer: notify user of --update-refs activity 2022-07-15 13:20 ` Derrick Stolee @ 2022-07-16 20:51 ` Elijah Newren 0 siblings, 0 replies; 144+ messages in thread From: Elijah Newren @ 2022-07-16 20:51 UTC (permalink / raw) To: Derrick Stolee Cc: Phillip Wood, Derrick Stolee via GitGitGadget, Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler On Fri, Jul 15, 2022 at 6:20 AM Derrick Stolee <derrickstolee@github.com> wrote: > > On 7/15/2022 6:12 AM, Phillip Wood wrote: > > > Thanks for the comprehensive commit message and for implementing an excellent suggestion from Elijah. I wonder if it makes sense to distinguish between the current branch and all the others when writing the update message as we do here or if all the refs should just be in a single list. I also think it doesn't matter much and we can change it later if we want. > > I'm definitely open to suggestions, but I also think we should start > somewhere and see what users think. Since the mechanisms for updating the > refs are different, I felt it was appropriate to have different error > messages. I think the separate messages are fine, but I'm a little surprised by the wording of this rationale. The mechanisms for updating the refs are implementational details that are likely not known by the end user, and aren't something they can tweak either. As such, I don't think the mechanism used should result in any differences in end-user messages here. However, one difference here is that one of the refs' changes (namely, the one corresponding to HEAD) will also have its changes be reflected in the current working directory and index. That may be a good reason to keep its update message separate from the rest of the refs. But on a related tangent... I'm still curious if people are going to be surprised to not see an "update-ref" line in the todo list for the HEAD ref; I was at first, and only understood the rationale for excluding it based on the implementational details. I know as far as the implementation goes that if such a line did appear in the todo list, then you'd probably just ignore it (other then ensuring it was the last line) and then let the pre-existing mechanisms in rebase update the HEAD ref...but even with that implementation "weirdness", I think incorporating it might be less surprising to the end user than the current behavior. Or maybe I'm just knee-deep in implementation details anyway, and I'm just guessing what others might think. I don't feel strongly about it, just thought I'd surface the thought. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 12/12] sequencer: notify user of --update-refs activity 2022-07-12 13:07 ` [PATCH v4 12/12] sequencer: notify user of --update-refs activity Derrick Stolee via GitGitGadget 2022-07-15 10:12 ` Phillip Wood @ 2022-07-16 22:09 ` Elijah Newren 2022-07-19 16:09 ` Derrick Stolee 1 sibling, 1 reply; 144+ messages in thread From: Elijah Newren @ 2022-07-16 22:09 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood, Derrick Stolee On Tue, Jul 12, 2022 at 6:07 AM Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com> wrote: > > From: Derrick Stolee <derrickstolee@github.com> > > When the user runs 'git rebase -i --update-refs', the end message still > says only > > Successfully rebased and updated <HEAD-ref>. > > Update the sequencer to collect the successful (and unsuccessful) ref > updates due to the --update-refs option, so the end message now says > > Successfully rebased and updated <HEAD-ref>. > Updated the following refs with --update-refs: > refs/heads/first > refs/heads/third This seems good. > Failed to update the following refs with --update-refs: > refs/heads/second This is good, but I think it could be improved. Could we also include the commit to which rebase would have updated the branch to? That would allow the user to manually update it if they want, or at least see a range-diff between what we would have updated it to and what it now has. Without that information, the user might have difficulty correcting that branch. Anyway, I've only been able to find a few minor things in reviewing this series. It looks really good overall, and I'm sure lots of people will be making use of it! ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 12/12] sequencer: notify user of --update-refs activity 2022-07-16 22:09 ` Elijah Newren @ 2022-07-19 16:09 ` Derrick Stolee 0 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee @ 2022-07-19 16:09 UTC (permalink / raw) To: Elijah Newren, Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood On 7/16/2022 6:09 PM, Elijah Newren wrote: > On Tue, Jul 12, 2022 at 6:07 AM Derrick Stolee via GitGitGadget > <gitgitgadget@gmail.com> wrote: >> >> From: Derrick Stolee <derrickstolee@github.com> >> >> When the user runs 'git rebase -i --update-refs', the end message still >> says only >> >> Successfully rebased and updated <HEAD-ref>. >> >> Update the sequencer to collect the successful (and unsuccessful) ref >> updates due to the --update-refs option, so the end message now says >> >> Successfully rebased and updated <HEAD-ref>. >> Updated the following refs with --update-refs: >> refs/heads/first >> refs/heads/third > > This seems good. > >> Failed to update the following refs with --update-refs: >> refs/heads/second > > This is good, but I think it could be improved. Could we also include > the commit to which rebase would have updated the branch to? That > would allow the user to manually update it if they want, or at least > see a range-diff between what we would have updated it to and what it > now has. Without that information, the user might have difficulty > correcting that branch. Would you mind if I left this as something for #leftoverbits? I expect that a follow-up series will be necessary once we have more user feedback. This isn't the only item delayed until after more feedback. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 00/12] rebase: update branches in multi-part topic 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (11 preceding siblings ...) 2022-07-12 13:07 ` [PATCH v4 12/12] sequencer: notify user of --update-refs activity Derrick Stolee via GitGitGadget @ 2022-07-12 15:37 ` Junio C Hamano 2022-07-14 14:50 ` Derrick Stolee 2022-07-15 15:41 ` Phillip Wood 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget 14 siblings, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-07-12 15:37 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, Derrick Stolee "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > This series is based on ds/branch-checked-out. > ... > Updates in v4 > ============= > > This version took longer than I'd hoped (I had less time to work on it than > anticipated) but it also has some major updates. These major updates are > direct responses to the significant review this series has received. Thank > you! > > * The update-refs file now stores "ref/before/after" triples (still > separated by lines). This allows us to store the "before" OID of a ref in > addition to the "after" that we will write to that ref at the end of the > rebase. This allows us to do a "force-with-lease" update. The > branch_checked_out() updates should prevent Git from updating those refs > while under the rebase, but older versions and third-party tools don't > have that protection. Nice. > * The update-refs file is updated with every update to the todo-list file. > This allows for some advanced changes to the file, including removing, > adding, and duplicating 'update-ref' commands. > * The message at the end of the rebase process now lists which refs were > updated with the update-ref steps. This includes any ref updates that > fail. > * The branch_checked_out() tests now use 'git bisect' and 'git rebase' as > black-boxes instead of testing their internals directly. > > Here are the more minor updates: > > * Dropped an unnecessary stat() call. > * Updated commit messages to include extra details, based on confusion in > last round. > * The HEAD branch no longer appears as a comment line in the initial todo > list. > * The update-refs file is now written using a lockfile. > * Tests now use test_cmp_rev. > * A memory leak ('path' variable) is resolved. Interesting. This is a tangent, but may serve as some food for thought. When I queue (or develop myself) a topic that depends on another topic, I often do $ git checkout --detach vX.Y.Z ;# choose an appropriate base $ git merge --into derived-topic base-topic $ develop develop (or "git am") which would end up in vX.Y.Z -----M---A---B---C derived-topic / base-topic so that "git log --first-parent --oneline master.." would show the commits in the topic plus the fact that it depends on which other topic recorded in a single merge commit. A topic that depends on two or more topics can be handled the same way. One good thing about this arrangement, unlike the "totally linear" one depicted at the top of your cover letter, is that it is easier to rebuild each topic independently and the first-parent view is still useful. If you futz with the base topic in a totally linear history, "log --decorate" of the derived topic would no longer tell you where the old iteration of the base topic ended. It would be very nice to see the update-ref feature (or something like that) makes it easy to deal with such a topology, too. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 00/12] rebase: update branches in multi-part topic 2022-07-12 15:37 ` [PATCH v4 00/12] rebase: update branches in multi-part topic Junio C Hamano @ 2022-07-14 14:50 ` Derrick Stolee 2022-07-14 18:11 ` Junio C Hamano 2022-07-16 20:56 ` Elijah Newren 0 siblings, 2 replies; 144+ messages in thread From: Derrick Stolee @ 2022-07-14 14:50 UTC (permalink / raw) To: Junio C Hamano, Derrick Stolee via GitGitGadget Cc: git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren [-- Attachment #1: Type: text/plain, Size: 3976 bytes --] On 7/12/22 11:37 AM, Junio C Hamano wrote:> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > This is a tangent, but may serve as some food for thought. > > When I queue (or develop myself) a topic that depends on another > topic, I often do > > $ git checkout --detach vX.Y.Z ;# choose an appropriate base > $ git merge --into derived-topic base-topic > $ develop develop (or "git am") > > which would end up in > > vX.Y.Z -----M---A---B---C derived-topic > / > base-topic > > so that "git log --first-parent --oneline master.." would show the > commits in the topic plus the fact that it depends on which other > topic recorded in a single merge commit. A topic that depends on > two or more topics can be handled the same way. > > One good thing about this arrangement, unlike the "totally linear" > one depicted at the top of your cover letter, is that it is easier > to rebuild each topic independently and the first-parent view is > still useful. If you futz with the base topic in a totally linear > history, "log --decorate" of the derived topic would no longer tell > you where the old iteration of the base topic ended. > > It would be very nice to see the update-ref feature (or something > like that) makes it easy to deal with such a topology, too. Your topology is an interesting one, but --update-refs isn't limited to linear history. (Begin aside) One goal of this feature is to make it easier to manage the topics that are being juggled in friendly forks. For example, git-for-windows/git and microsoft/git have some topics that evolve version-to-version but might be cleaned up and sent upstream. Both of these forks use 'git rebase --rebase-merges=rebase-cousins' when consuming new upstream versions to keep the merge structure of these new topics. However, we lose the branch names and need to reconstruct them from the context of the merge commits. With --update-refs, we can automatically rewrite the branches that are included in these individual topics. That might make it simpler to extract a series to send upstream. One test in this series does test such a case with the --rebase-merges option. (End aside) Back to your topology, I wonder what your rebase command looks like when tracking those topics. The goal of --update-refs is to help rebase multiple branches at the same time, and with your example here, it would imply you want to rebase both dependent topics. Before: A---B----C---M---Q1---Q2---Q3 <-- refs/heads/Q \ / P1--P2--P3 <-- refs/heads/P After rebasing both topics simultaneously (with 'git rebase --update-refs C' while Q is checked out): A---B---C---D---P1---P2---P3---Q1---Q2---Q3 ^ ^ refs/heads/P refs/heads/Q But it seems what you mean to say is to update the merge commit M, which means that the 'P' branch above has been updated independently of the 'Q' branch, so we need to update 'Q' after-the-fact. I'm not sure what that rebase would look like, indepdendent of updating refs. Do you have an example rebase command that manipulates the commits the way you want? Then I can better understand how the --update-refs could fit in with that. (Or maybe the point of your tangent is that there isn't an option.) If instead we thought about an example like re-rolling the 'next' branch entirely on top of the 'master' branch, then we have an example that is closer to the friendly fork example. (I know this isn't a realistic scenario since we don't rewrite the commits already merged to 'next', but it's an interesting stress test.) While having 'next' checked out, I ran git rebase -i --update-refs --rebase-merges=rebase-cousins master and it updated all of the branches currently in the region master..next. I've attached the output of git -c log.excludeDecoration="refs/remotes/*" \ log --oneline --graph --boundary master..next just to show what this looks like. Thanks, -Stolee [-- Attachment #2: rebase-result.txt --] [-- Type: text/plain, Size: 13575 bytes --] * bdf2d8e9e1 Merge branch 'll/curl-accept-language' into next |\ | * b0c4adcdd7 remote-curl: send Accept-Language header to server * | 36fcf76053 Merge branch 'jk/diff-files-cleanup-fix' into next |\ \ | * | 04393ae7f7 diff-files: move misplaced cleanup label * | | 3f6ccaf58c Merge branch 'rs/cocci-array-copy' into next |\ \ \ | * | | dc574132a0 cocci: avoid normalization rules for memcpy * | | | 2125d3847d Merge branch 'jk/ref-filter-discard-commit-buffer' into next |\ \ \ \ | * | | | 359b01ca84 ref-filter: disable save_commit_buffer while traversing * | | | | fd0e10a90f Merge branch 'jk/clone-unborn-confusion' into next |\ \ \ \ \ | * | | | | 2eb5d38386 clone: move unborn head creation to update_head() | * | | | | 73a0858224 clone: use remote branch if it matches default HEAD | * | | | | 54a0c1896c clone: propagate empty remote HEAD even with other branches | * | | | | 51fa61ba3e clone: drop extra newline from warning message | | |/ / / | |/| | | * | | | | 05d6fbe0e5 Merge branch 'hx/lookup-commit-in-graph-fix' into next |\ \ \ \ \ | * | | | | 854d62d9fe t5330: remove run_with_limited_processses() * | | | | | 5386228a28 Merge branch 'jc/resolve-undo' into next |\ \ \ \ \ \ | * | | | | | b237e81516 fsck: do not dereference NULL while checking resolve-undo data * | | | | | | a34a4435d9 Merge branch 'sg/multi-pack-index-parse-options-fix' into next |\ \ \ \ \ \ \ | * | | | | | | cc74afb83f multi-pack-index: simplify handling of unknown --options | | |_|_|/ / / | |/| | | | | * | | | | | | 57b36b2e9b Merge branch 'bc/nettle-sha256' into next |\ \ \ \ \ \ \ | * | | | | | | e555735836 sha256: add support for Nettle | |/ / / / / / * | | | | | | 11cf608d10 Merge branch 'kk/p4-client-name-encoding-fix' into next |\ \ \ \ \ \ \ | * | | | | | | 34f67c9619 git-p4: fix bug with encoding of p4 client name | |/ / / / / / * | | | | | | 99a9342263 Merge branch 'jd/gpg-interface-trust-level-string' into next |\ \ \ \ \ \ \ | * | | | | | | 803978da49 gpg-interface: add function for converting trust level to string | |/ / / / / / * | | | | | | 47942e9ac7 Merge branch 'ab/cocci-unused' into next |\ \ \ \ \ \ \ | * | | | | | | 06f5f8940c cocci: generalize "unused" rule to cover more than "strbuf" | * | | | | | | 4f40f6cb73 cocci: add and apply a rule to find "unused" strbufs | * | | | | | | 7a9a10b10e cocci: have "coccicheck{,-pending}" depend on "coccicheck-test" | * | | | | | | f7ff6597a7 cocci: add a "coccicheck-test" target and test *.cocci rules | * | | | | | | af0aa6904b Makefile & .gitignore: ignore & clean "git.res", not "*.res" | * | | | | | | 7b63ea5750 Makefile: remove mandatory "spatch" arguments from SPATCH_FLAGS * | | | | | | | 9720950472 Merge branch 'gc/submodule-use-super-prefix' into next |\ \ \ \ \ \ \ \ | * | | | | | | | 5ad87271cf submodule--helper: remove display path helper | * | | | | | | | d7a714fddc submodule--helper update: use --super-prefix | * | | | | | | | b0f8b21305 submodule--helper: remove unused SUPPORT_SUPER_PREFIX flags | * | | | | | | | 58cec298f1 submodule--helper: use correct display path helper | * | | | | | | | cb49e1e8d3 submodule--helper: don't recreate recursive prefix | * | | | | | | | 618b8445d9 submodule--helper update: use display path helper | * | | | | | | | 8fc36c39d9 submodule--helper tests: add missing "display path" coverage | * | | | | | | | c9e221b124 Merge branch 'ab/submodule-cleanup' into gc/submodule-use-super-prefix | |\ \ \ \ \ \ \ \ | | |_|/ / / / / / | |/| | | | | | | * | | | | | | | | 172d4d2a2b Merge branch 'en/merge-dual-dir-renames-fix' into next |\ \ \ \ \ \ \ \ \ | * | | | | | | | | 8fe673ee1d merge-ort: fix issue with dual rename and add/add conflict | * | | | | | | | | 2ef5b00eba merge-ort: shuffle the computation and cleanup of potential collisions | * | | | | | | | | b3468c2055 merge-ort: make a separate function for freeing struct collisions | * | | | | | | | | a4719d9e46 merge-ort: small cleanups of check_for_directory_rename | * | | | | | | | | 70fae629e6 t6423: add tests of dual directory rename plus add/add conflict | | |_|_|_|_|/ / / | |/| | | | | | | * | | | | | | | | 9fc51f4bdd Merge branch 'ab/test-without-templates' into next |\ \ \ \ \ \ \ \ \ | * | | | | | | | | c8ab2aa9ce tests: don't assume a .git/info for .git/info/sparse-checkout | * | | | | | | | | 913769a360 tests: don't assume a .git/info for .git/info/exclude | * | | | | | | | | af4eb37df7 tests: don't assume a .git/info for .git/info/refs | * | | | | | | | | 5383f7f9de tests: don't assume a .git/info for .git/info/attributes | * | | | | | | | | dc7e8c3b3a tests: don't assume a .git/info for .git/info/grafts | * | | | | | | | | fdc489c66c tests: don't depend on template-created .git/branches | * | | | | | | | | 7a1cb29f61 t0008: don't rely on default ".git/info/exclude" | |/ / / / / / / / * | | | | | | | | 8d446e7df8 Merge branch 'ab/build-gitweb' into next |\ \ \ \ \ \ \ \ \ | * | | | | | | | | a35258c62a gitweb/Makefile: add a "NO_GITWEB" parameter | * | | | | | | | | d3b827408c Makefile: build 'gitweb' in the default target | * | | | | | | | | affc3b755c gitweb/Makefile: include in top-level Makefile | * | | | | | | | | 27438ef5e0 gitweb: remove "test" and "test-installed" targets | * | | | | | | | | b82d66eb0c gitweb/Makefile: prepare to merge into top-level Makefile | * | | | | | | | | 564ebde3d3 gitweb/Makefile: clear up and de-duplicate the gitweb.{css,js} vars | * | | | | | | | | 1e08fa5e2b gitweb/Makefile: add a $(GITWEB_ALL) variable | * | | | | | | | | 7decdb9b4a gitweb/Makefile: define all .PHONY prerequisites inline * | | | | | | | | | 3ab689137a Merge branch 'ab/test-tool-leakfix' into next |\ \ \ \ \ \ \ \ \ \ | * | | | | | | | | | f40a693450 test-tool delta: fix a memory leak | * | | | | | | | | | 34e691288d test-tool ref-store: fix a memory leak | * | | | | | | | | | 9794633b4e test-tool bloom: fix memory leaks | * | | | | | | | | | 1caaa858cc test-tool json-writer: fix memory leaks | * | | | | | | | | | a20b0dc796 test-tool regex: call regfree(), fix memory leaks | * | | | | | | | | | 1c343e5aef test-tool urlmatch-normalization: fix a memory leak | * | | | | | | | | | 9afa46d4a6 test-tool {dump,scrap}-cache-tree: fix memory leaks | * | | | | | | | | | e287a5b0a4 test-tool path-utils: fix a memory leak | * | | | | | | | | | 330ca8501b test-tool test-hash: fix a memory leak | | |_|/ / / / / / / | |/| | | | | | | | * | | | | | | | | | b9183a05c5 Merge branch 'ab/leakfix' into next |\ \ \ \ \ \ \ \ \ \ | * | | | | | | | | | ece3974ba6 pull: fix a "struct oid_array" memory leak | * | | | | | | | | | 27472b5195 cat-file: fix a common "struct object_context" memory leak | * | | | | | | | | | 55916bba0f gc: fix a memory leak | * | | | | | | | | | 33d0dda633 checkout: avoid "struct unpack_trees_options" leak | * | | | | | | | | | e72e12cc02 merge-file: fix memory leaks on error path | * | | | | | | | | | 480a0e30a7 merge-file: refactor for subsequent memory leak fix | * | | | | | | | | | d90dafbe31 cat-file: fix a memory leak in --batch-command mode | * | | | | | | | | | fd74ac95ac revert: free "struct replay_opts" members | * | | | | | | | | | bc57ba1d54 submodule.c: free() memory from xgetcwd() | * | | | | | | | | | 74a06a9f21 clone: fix memory leak in wanted_peer_refs() | * | | | | | | | | | 99b6c45d8f check-ref-format: fix trivial memory leak | |/ / / / / / / / / * | | | | | | | | | 542f8fec7e Merge branch 'jc/builtin-mv-move-array' into next |\ \ \ \ \ \ \ \ \ \ | * | | | | | | | | | 4901a36b4b builtin/mv.c: use the MOVE_ARRAY() macro instead of memmove() | | |_|/ / / / / / / | |/| | | | | | | | * | | | | | | | | | 1b5d8c02cf Merge branch 'fr/vimdiff-layout-fix' into next |\ \ \ \ \ \ \ \ \ \ | * | | | | | | | | | a62efbaabb vimdiff: make layout engine more robust against user vim settings | |/ / / / / / / / / * | | | | | | | | | f022414e31 Merge branch 'hx/lookup-commit-in-graph-fix' into next |\ \ \ \ \ \ \ \ \ \ | | |_|_|_|_|_|/ / / | |/| | | | | | | | | * | | | | | | | | a4d60a6221 commit-graph.c: no lazy fetch in lookup_commit_in_graph() | |/ / / / / / / / * | | | | | | | | 90790c4455 Merge branch 'ab/submodule-cleanup' into next |\ \ \ \ \ \ \ \ \ | | |_|_|/ / / / / | |/| | | | | | | | * | | | | | | | 5b893f7d81 git-sh-setup.sh: remove "say" function, change last users | * | | | | | | | 2eec463739 git-submodule.sh: use "$quiet", not "$GIT_QUIET" | * | | | | | | | b788fc671b submodule--helper: eliminate internal "--update" option | * | | | | | | | 8f12108c29 submodule--helper: understand --checkout, --merge and --rebase synonyms | * | | | | | | | 36d45163b6 submodule--helper: report "submodule" as our name in some "-h" output | * | | | | | | | 6e556c412e submodule--helper: rename "absorb-git-dirs" to "absorbgitdirs" | * | | | | | | | 0d68ee723e submodule update: remove "-v" option | * | | | | | | | d9c7f69aaa submodule--helper: have --require-init imply --init | * | | | | | | | da3aae9e84 git-submodule.sh: remove unused top-level "--branch" argument | * | | | | | | | 757d092797 git-submodule.sh: make the "$cached" variable a boolean | * | | | | | | | 960fad98e8 git-submodule.sh: remove unused $prefix variable | * | | | | | | | 85775255f1 git-submodule.sh: remove unused sanitize_submodule_env() * | | | | | | | | d1ce5a9649 Merge branch 'sy/mv-out-of-cone' into next |\ \ \ \ \ \ \ \ \ | * | | | | | | | | b91a2b6594 mv: add check_dir_in_index() and solve general dir check issue | * | | | | | | | | 24ea81d9ac mv: use flags mode for update_mode | * | | | | | | | | 8a26a3915f mv: check if <destination> exists in index to handle overwriting | * | | | | | | | | 6645b03ca5 mv: check if out-of-cone file exists in index with SKIP_WORKTREE bit | * | | | | | | | | 7889755bae mv: decouple if/else-if checks using goto | * | | | | | | | | 707fa2f76a mv: update sparsity after moving from out-of-cone to in-cone | * | | | | | | | | 1143cc01b7 t1092: mv directory from out-of-cone to in-cone | * | | | | | | | | 367844e5b7 t7002: add tests for moving out-of-cone file/directory | | |_|/ / / / / / | |/| | | | | | | * | | | | | | | | 97316a96ed Merge branch 'hx/unpack-streaming' into next |\ \ \ \ \ \ \ \ \ | * | | | | | | | | aaf81223f4 unpack-objects: use stream_loose_object() to unpack large objects | * | | | | | | | | 3c3ca0b0c1 core doc: modernize core.bigFileThreshold documentation | * | | | | | | | | 2b6070ac4c object-file.c: add "stream_loose_object()" to handle large object | * | | | | | | | | 21e7d88140 object-file.c: factor out deflate part of write_loose_object() | * | | | | | | | | 97a9db6ffb object-file.c: refactor write_loose_object() to several steps | * | | | | | | | | a1bf5ca29f unpack-objects: low memory footprint for get_data() in dry_run mode * | | | | | | | | | b4846d0daa Merge branch 'en/merge-tree' into next |\ \ \ \ \ \ \ \ \ \ | * | | | | | | | | | 7260e87248 git-merge-tree.txt: add a section on potentional usage mistakes | * | | | | | | | | | 7976721d17 merge-tree: add a --allow-unrelated-histories flag | * | | | | | | | | | 7c48b27822 merge-tree: allow `ls-files -u` style info to be NUL terminated | * | | | | | | | | | de90581141 merge-ort: optionally produce machine-readable output | * | | | | | | | | | cb2607759e merge-ort: store more specific conflict information | * | | | | | | | | | 2715e8a931 merge-ort: make `path_messages` a strmap to a string_list | * | | | | | | | | | 6debb7527b merge-ort: store messages in a list, not in a single strbuf | * | | | | | | | | | b520bc6caa merge-tree: provide easy access to `ls-files -u` style info | * | | | | | | | | | 7fa3338870 merge-tree: provide a list of which files have conflicts | * | | | | | | | | | a4040cfa35 merge-ort: remove command-line-centric submodule message from merge-ort | * | | | | | | | | | fae26ce79c merge-ort: provide a merge_get_conflicted_files() helper function | * | | | | | | | | | a1a7811975 merge-tree: support including merge messages in output | * | | | | | | | | | a34edae68a merge-ort: split out a separate display_update_messages() function | * | | | | | | | | | 1f0c3a29da merge-tree: implement real merges | * | | | | | | | | | 6ec755a0e1 merge-tree: add option parsing and initial shell for real merge function | * | | | | | | | | | 55e48f6bf7 merge-tree: move logic for existing merge into new function | * | | | | | | | | | 70176b7015 merge-tree: rename merge_trees() to trivial_merge_trees() * | | | | | | | | | | d340a03e89 Merge branch 'gg/worktree-from-the-above' into next |\ \ \ \ \ \ \ \ \ \ \ | * | | | | | | | | | | 299f88a3b3 dir: minor refactoring / clean-up | * | | | | | | | | | | 992bb2eff9 dir: traverse into repository | | |_|_|_|/ / / / / / | |/| | | | | | | | | * | | | | | | | | | | a06ded0851 Merge branch 'jc/resolve-undo' into next |\ \ \ \ \ \ \ \ \ \ \ | |/ / / / / / / / / / |/| | | | | | | / / / | | |_|_|_|_|_|/ / / | |/| | | | | | | | | * | | | | | | | | 0c2d8a5b28 revision: mark blobs needed for resolve-undo as reachable |/ / / / / / / / / | o / / / / / / / f770e9f396 Git 2.37-rc2 | / / / / / / / | o / / / / / / ab336e8f1c Seventh batch | / / / / / / | | o / / / / 8168d5e9c2 Git 2.37-rc0 | | / / / / | | o / / / 2668e3608e Sixth batch | | / / / | | o / / 30cc8d0f14 A regression fix for 2.37 | | / / | o / / e4a4b31577 Git 2.37 | / / o / / 4e2a4d1dd4 The second batch / / o / 54c8a7c379 revisions API: add a TODO for diff_free(&revs->diffopt) / o 1e59178e3f Sync with 'maint' ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 00/12] rebase: update branches in multi-part topic 2022-07-14 14:50 ` Derrick Stolee @ 2022-07-14 18:11 ` Junio C Hamano 2022-07-16 21:23 ` Elijah Newren 2022-07-16 20:56 ` Elijah Newren 1 sibling, 1 reply; 144+ messages in thread From: Junio C Hamano @ 2022-07-14 18:11 UTC (permalink / raw) To: Derrick Stolee Cc: Derrick Stolee via GitGitGadget, git, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren Derrick Stolee <derrickstolee@github.com> writes: > Before: > > A---B----C---M---Q1---Q2---Q3 <-- refs/heads/Q > \ / > P1--P2--P3 <-- refs/heads/P > > After rebasing both topics simultaneously (with 'git rebase --update-refs > C' while Q is checked out): > > A---B---C---D---P1---P2---P3---Q1---Q2---Q3 > ^ ^ > refs/heads/P refs/heads/Q > > But it seems what you mean to say is to update the merge commit M, which > means that the 'P' branch above has been updated independently of the 'Q' > branch, so we need to update 'Q' after-the-fact. I am not aiming to flatten P and Q into a single strand of pearls. That would defeat the point of "git log --oneline --first-parent" that can be used to view "master..Q", whose output would be "at the bottom the topic P lies there, and on top there are 3 patches". P's may be your ds/branch-checked-out topic while Q's may be this topic. Other people may find bugs, improvements and a room for unwanted churns in the former, and P may gain a few more commit, in which case M thru Q3 needs to be rebuilt. In a manual procedure, when I realize that P will gain a few more patches (or gets rewritten): * find what other topics depend on P and make a mental note (i.e. Q needs to be rebuilt) * perform an equivalent of "git rebase -i --onto A P", but without using "git rebase". - git checkout P - git checkout --detach master... ;# reuse the same base - git am -s ;# apply - git rebase -i ;# minor fix-up while queuing - git range-diff @{-1}... ;# sanity check - make test ;# further sanity check - git checkout -B @{-1} ;# update P to the new round - git range-diff @{1}... ;# final sanity check this rebuilds P1, P2, and P3 into a new series on A * for each topic that needs rebuilding (i.e. Q), find M and rebuild it - git checkout Q - git checkout --detach master... ;# reuse the same base - git merge --into Q P ;# recreate M with updated P - git rebase --onto HEAD M Q ;# rebuild Q - git checkout -B @{-1} ;# update Q to sit on top of new P - git range-diff @{1}... ;# sanity check (should be empty) > I'm not sure what that rebase would look like, indepdendent of > updating refs. I suspect that a creative use of "git rebase --rebase-merge master Q" should allow me to get there. Here is an outline of the todo list you'd get out of "git rebase --rebase-merge -i v2.37.0" while the topic ds/rebase-update-ref is checked out: ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- label onto # Branch ds-branch-checked-out reset 5ed49a75f3 # Merge branch 'os/fetch-check-not-current-branch' pick 31ad6b61bd branch: add branch_checked_out() helper pick d2ba271aad branch: check for bisects and rebases ... pick 9bef0b1e6e branch: drop unused worktrees variable label ds-branch-checked-out reset onto merge -C 7fefa1b68e ds-branch-checked-out # Merge branch 'ds/branch-checked-out' into ds/rebase-update-ref pick a0bfa0ec53 t2407: test bisect and rebase as black-boxes pick 43547f7a52 t2407: test branches currently using apply backend ... pick 8b2a776cab sequencer: notify user of --update-refs activity ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- So it would likely involve - "git checkout Q && git rebase -i --rebase-merge master" to get the todo list. - remove all "pick"s for existing commits on branch P, and replace them with a "break" - add an update-ref insn to update ds/branch-checked-out topic using the ds-branch-checked-out label at the end of the todo list. - exit the editor, and in the "break" session, run "am" to accept the new round of patches for P. - "git rebase --continue" to let the tip labeled as P and let the rest of the todo list rebuild Q but I am not sure what should happen when there are more than one dependent topic (i.e. in addition to Q, topic R also depends on P). It also is unclear in the above procedure with "rebase-merge" what to feed to "range-diff" in the sanity-checking step. I could type "git range-diff P..." but being able to use @{-1} is a lot handier. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 00/12] rebase: update branches in multi-part topic 2022-07-14 18:11 ` Junio C Hamano @ 2022-07-16 21:23 ` Elijah Newren 0 siblings, 0 replies; 144+ messages in thread From: Elijah Newren @ 2022-07-16 21:23 UTC (permalink / raw) To: Junio C Hamano Cc: Derrick Stolee, Derrick Stolee via GitGitGadget, Git Mailing List, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood On Thu, Jul 14, 2022 at 11:11 AM Junio C Hamano <gitster@pobox.com> wrote: > > Derrick Stolee <derrickstolee@github.com> writes: > > > Before: > > > > A---B----C---M---Q1---Q2---Q3 <-- refs/heads/Q > > \ / > > P1--P2--P3 <-- refs/heads/P > > > > After rebasing both topics simultaneously (with 'git rebase --update-refs > > C' while Q is checked out): > > > > A---B---C---D---P1---P2---P3---Q1---Q2---Q3 > > ^ ^ > > refs/heads/P refs/heads/Q > > > > But it seems what you mean to say is to update the merge commit M, which > > means that the 'P' branch above has been updated independently of the 'Q' > > branch, so we need to update 'Q' after-the-fact. > > I am not aiming to flatten P and Q into a single strand of pearls. > That would defeat the point of "git log --oneline --first-parent" > that can be used to view "master..Q", whose output would be "at the > bottom the topic P lies there, and on top there are 3 patches". > > P's may be your ds/branch-checked-out topic while Q's may be this > topic. Other people may find bugs, improvements and a room for > unwanted churns in the former, and P may gain a few more commit, > in which case M thru Q3 needs to be rebuilt. > > In a manual procedure, when I realize that P will gain a few > more patches (or gets rewritten): > > * find what other topics depend on P and make a mental note (i.e. Q > needs to be rebuilt) > > * perform an equivalent of "git rebase -i --onto A P", but without > using "git rebase". > > - git checkout P > - git checkout --detach master... ;# reuse the same base > - git am -s ;# apply > - git rebase -i ;# minor fix-up while queuing > - git range-diff @{-1}... ;# sanity check > - make test ;# further sanity check > - git checkout -B @{-1} ;# update P to the new round > - git range-diff @{1}... ;# final sanity check > > this rebuilds P1, P2, and P3 into a new series on A > > * for each topic that needs rebuilding (i.e. Q), find M and rebuild > it > > - git checkout Q > - git checkout --detach master... ;# reuse the same base > - git merge --into Q P ;# recreate M with updated P > - git rebase --onto HEAD M Q ;# rebuild Q > - git checkout -B @{-1} ;# update Q to sit on top of new P > - git range-diff @{1}... ;# sanity check (should be empty) Thanks for sharing all these details, and the ones below. > > I'm not sure what that rebase would look like, indepdendent of > > updating refs. > > I suspect that a creative use of "git rebase --rebase-merge master > Q" should allow me to get there. Here is an outline of the todo > list you'd get out of "git rebase --rebase-merge -i v2.37.0" while > the topic ds/rebase-update-ref is checked out: > > ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- > label onto > > # Branch ds-branch-checked-out > reset 5ed49a75f3 # Merge branch 'os/fetch-check-not-current-branch' > pick 31ad6b61bd branch: add branch_checked_out() helper > pick d2ba271aad branch: check for bisects and rebases > ... > pick 9bef0b1e6e branch: drop unused worktrees variable > label ds-branch-checked-out > > reset onto > merge -C 7fefa1b68e ds-branch-checked-out # Merge branch 'ds/branch-checked-out' into ds/rebase-update-ref > pick a0bfa0ec53 t2407: test bisect and rebase as black-boxes > pick 43547f7a52 t2407: test branches currently using apply backend > ... > pick 8b2a776cab sequencer: notify user of --update-refs activity > ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- > > So it would likely involve > > - "git checkout Q && git rebase -i --rebase-merge master" to get > the todo list. > > - remove all "pick"s for existing commits on branch P, and replace > them with a "break" > > - add an update-ref insn to update ds/branch-checked-out topic > using the ds-branch-checked-out label at the end of the todo > list. > > - exit the editor, and in the "break" session, run "am" to accept > the new round of patches for P. > > - "git rebase --continue" to let the tip labeled as P and let the > rest of the todo list rebuild Q > > but I am not sure what should happen when there are more than one > dependent topic (i.e. in addition to Q, topic R also depends on P). Yeah, rebase's hard-coded HEAD assumption and including it in an implicit range, makes it hard to get more general revision range expressions. If we had something that just let you specify your revision range, then you could handle multiple topics, using syntax something like: git replay --keep-base ^master P Q R and then edit the resulting todo file (where --interactive and an equivalent of --rebase-merges are both assumed). You don't even necessarily have to have any of P, Q, or R checked out. In fact, if we can use revision ranges, then perhaps it'd be even nicer to run git replay --keep-base --contained --ancestry-path P..seen (where --contained says update all refs pointing to any commit contained in the revision range, instead of just the explicitly listed refs). This would also remove all the picks for P for you, since it's the left side of that range. So, you'd just need to adjust the first "reset" directive to an older commit, and add a "break" directive right after it followed by an "update-ref" directive for P. During the "break", you can "git-am" the new patches to accept the new version of P. You wouldn't have to make a mental note of any other branches (Q, R, S, etc.) that also need to be included, since they are contained in seen and the --ancestry-path gets you just the right range of commits, replaying them all including merges (and I've got plans for improving the replaying of merges too). However... What if someone wants to edit the patches in topic P too, rather than just eject and replace them? There's no <revision range> I know of to get that if the base of P is some very common commit, because "--ancestry-path master..seen" will suddenly balloon the number of topics involved rather dramatically. I guess you could just take that todo list and edit it (since most commits in the list would just be replayed on the same base and end up fast-forwarding and thus not be changed; only commits the user edited and their descendants would change), but that todo list could be unworkably long to meddle with. I'm curious if anyone has ideas for that. > It also is unclear in the above procedure with "rebase-merge" what > to feed to "range-diff" in the sanity-checking step. I could type > "git range-diff P..." but being able to use @{-1} is a lot handier. I think you could use "git range-diff seen@{1}...seen" after the operation I outlined above. Granted, that only helps sanity check the non-merge commits, since range-diff currently ignores merges, but perhaps if we teach range-diff to make use of --diff-merges=remerge then it could? ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 00/12] rebase: update branches in multi-part topic 2022-07-14 14:50 ` Derrick Stolee 2022-07-14 18:11 ` Junio C Hamano @ 2022-07-16 20:56 ` Elijah Newren 1 sibling, 0 replies; 144+ messages in thread From: Elijah Newren @ 2022-07-16 20:56 UTC (permalink / raw) To: Derrick Stolee Cc: Junio C Hamano, Derrick Stolee via GitGitGadget, Git Mailing List, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood On Thu, Jul 14, 2022 at 7:50 AM Derrick Stolee <derrickstolee@github.com> wrote: > > On 7/12/22 11:37 AM, Junio C Hamano wrote:> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes: > > This is a tangent, but may serve as some food for thought. > > > > When I queue (or develop myself) a topic that depends on another > > topic, I often do > > > > $ git checkout --detach vX.Y.Z ;# choose an appropriate base > > $ git merge --into derived-topic base-topic > > $ develop develop (or "git am") > > > > which would end up in > > > > vX.Y.Z -----M---A---B---C derived-topic > > / > > base-topic > > > > so that "git log --first-parent --oneline master.." would show the > > commits in the topic plus the fact that it depends on which other > > topic recorded in a single merge commit. A topic that depends on > > two or more topics can be handled the same way. > > > > One good thing about this arrangement, unlike the "totally linear" > > one depicted at the top of your cover letter, is that it is easier > > to rebuild each topic independently and the first-parent view is > > still useful. If you futz with the base topic in a totally linear > > history, "log --decorate" of the derived topic would no longer tell > > you where the old iteration of the base topic ended. > > > > It would be very nice to see the update-ref feature (or something > > like that) makes it easy to deal with such a topology, too. > > Your topology is an interesting one, but --update-refs isn't limited to > linear history. > > (Begin aside) > > One goal of this feature is to make it easier to manage the topics that > are being juggled in friendly forks. For example, git-for-windows/git and > microsoft/git have some topics that evolve version-to-version but might be > cleaned up and sent upstream. Both of these forks use 'git rebase > --rebase-merges=rebase-cousins' when consuming new upstream versions to > keep the merge structure of these new topics. However, we lose the branch > names and need to reconstruct them from the context of the merge commits. > > With --update-refs, we can automatically rewrite the branches that are > included in these individual topics. That might make it simpler to extract > a series to send upstream. > > One test in this series does test such a case with the --rebase-merges > option. > > (End aside) > > Back to your topology, I wonder what your rebase command looks like when > tracking those topics. > > The goal of --update-refs is to help rebase multiple branches at the same > time, and with your example here, it would imply you want to rebase both > dependent topics. > > Before: > > A---B----C---M---Q1---Q2---Q3 <-- refs/heads/Q > \ / > P1--P2--P3 <-- refs/heads/P > > After rebasing both topics simultaneously (with 'git rebase --update-refs > C' while Q is checked out): > > A---B---C---D---P1---P2---P3---Q1---Q2---Q3 > ^ ^ > refs/heads/P refs/heads/Q > > But it seems what you mean to say is to update the merge commit M, which > means that the 'P' branch above has been updated independently of the 'Q' > branch, so we need to update 'Q' after-the-fact. I'm not sure what that > rebase would look like, indepdendent of updating refs. Since this is an aside, I'll take a chance to talk about stuff I'm working on. After git replay --onto main --contained main..Q (where "--contained" seems to be similar to rebase's --update-refs option, in that it assumes all refs pointing to history being rewritten should also be updated), or the more explicit git replay --onto main ^main P Q and assuming 'main' has one additional commit 'D' on top of 'C', you'd see A---B---C---D-----------M---Q1---Q2---Q3 <-- refs/heads/Q \ / P1--P2--P3 <-- refs/heads/P In other words, both P and Q would be replayed but with relative topology preserved. Also, important changes in M (conflict fixups or semantic fixes, etc.) would be preserved -- at least that's the plan. > Do you have an example rebase command that manipulates the commits the > way you want? Then I can better understand how the --update-refs could fit > in with that. (Or maybe the point of your tangent is that there isn't an > option.) This would be nice. > If instead we thought about an example like re-rolling the 'next' branch > entirely on top of the 'master' branch, then we have an example that is > closer to the friendly fork example. (I know this isn't a realistic > scenario since we don't rewrite the commits already merged to 'next', but > it's an interesting stress test.) > > While having 'next' checked out, I ran > > git rebase -i --update-refs --rebase-merges=rebase-cousins master Re-rolling 'next' might not be realistic, but re-rolling 'seen' isn't quite as far fetched. I think it'd be more likely to use no-rebase-cousins, so: git checkout seen && git rebase -i --update-refs [--rebase-merges=no-rebase-cousins] next or git replay -i --contained --keep-base next..seen > and it updated all of the branches currently in the region master..next. > I've attached the output of > > git -c log.excludeDecoration="refs/remotes/*" \ > log --oneline --graph --boundary master..next > > just to show what this looks like. One example that I thought might be useful, is tweaking a single topic in 'seen', and updating 'seen' to reflect those updates. This would be something like git replay --keep-base --first-parent ^next seen topic_branch Which would show you all the commits in the first-line history of next..topic_branch plus the first-line history of next..seen, and let you tweak that todo list. ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v4 00/12] rebase: update branches in multi-part topic 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (12 preceding siblings ...) 2022-07-12 15:37 ` [PATCH v4 00/12] rebase: update branches in multi-part topic Junio C Hamano @ 2022-07-15 15:41 ` Phillip Wood 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget 14 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-07-15 15:41 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, Derrick Stolee On 12/07/2022 14:06, Derrick Stolee via GitGitGadget wrote: I haven't looked at the first two patches but I think I read everything else that has changed in V4. It all looks good to me. I've left some comments on the later patches but I don't think there are any showstoppers and this version could be merged into next. Thanks for working on this Phillip > This series is based on ds/branch-checked-out. > > This is a feature I've wanted for quite a while. When working on the sparse > index topic, I created a long RFC that actually broke into three topics for > full review upstream. These topics were sequential, so any feedback on an > earlier one required updates to the later ones. I would work on the full > feature and use interactive rebase to update the full list of commits. > However, I would need to update the branches pointing to those sub-topics. > > This series adds a new --update-refs option to 'git rebase' (along with a > rebase.updateRefs config option) that adds 'update-ref' commands into the > TODO list. This is powered by the commit decoration machinery. > > As an example, here is my in-progress bundle URI RFC split into subtopics as > they appear during the TODO list of a git rebase -i --update-refs: > > pick 2d966282ff3 docs: document bundle URI standard > pick 31396e9171a remote-curl: add 'get' capability > pick 54c6ab70f67 bundle-uri: create basic file-copy logic > pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// > pick 6adaf842684 fetch: add --bundle-uri option > pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration > update-ref refs/heads/bundle-redo/fetch > > pick 1e3f6546632 clone: add --bundle-uri option > pick 9e4a6fe9b68 clone: --bundle-uri cannot be combined with --depth > update-ref refs/heads/bundle-redo/clone > > pick 5451cb6599c bundle-uri: create bundle_list struct and helpers > pick 3029c3aca15 bundle-uri: create base key-value pair parsing > pick a8b2de79ce8 bundle-uri: create "key=value" line parsing > pick 92625a47673 bundle-uri: unit test "key=value" parsing > pick a8616af4dc2 bundle-uri: limit recursion depth for bundle lists > pick 9d6809a8d53 bundle-uri: parse bundle list in config format > pick 287a732b54c bundle-uri: fetch a list of bundles > update-ref refs/heads/bundle-redo/list > > pick b09f8226185 protocol v2: add server-side "bundle-uri" skeleton > pick 520204dcd1c bundle-uri client: add minimal NOOP client > pick 62e8b457b48 bundle-uri client: add "git ls-remote-bundle-uri" > pick 00eae925043 bundle-uri: serve URI advertisement from bundle.* config > pick 4277440a250 bundle-uri client: add boolean transfer.bundleURI setting > pick caf4599a81d bundle-uri: allow relative URLs in bundle lists > pick df255000b7e bundle-uri: download bundles from an advertised list > pick d71beabf199 clone: unbundle the advertised bundles > pick c9578391976 t5601: basic bundle URI tests > # Ref refs/heads/bundle-redo/rfc-3 checked out at '/home/stolee/_git/git-bundles' > > update-ref refs/heads/bundle-redo/advertise > > > Here is an outline of the series: > > * Patch 1 updates some tests for branch_checked_out() to use 'git bisect' > and 'git rebase' as black-boxes instead of manually editing files inside > $GIT_DIR. (Thanks, Junio!) > * Patch 2 updates some tests for branch_checked_out() for the 'apply' > backend. > * Patch 3 updates branch_checked_out() to parse the > rebase-merge/update-refs file to block concurrent ref updates and > checkouts on branches selected by --update-refs. > * Patch 4 updates the todo list documentation to remove some unnecessary > dots in the 'merge' command. This makes it consistent with the 'fixup' > command before we document the 'update-ref' command. > * Patch 5 updates the definition of todo_command_info to use enum values as > array indices. > * Patches 6-8 implement the --update-refs logic itself. > * Patch 9 specifically updates the update-refs file every time the user > edits the todo-list (Thanks Phillip!) > * Patch 10 adds the rebase.updateRefs config option similar to > rebase.autoSquash. > * Patch 11 ignores the HEAD ref when creating the todo list instead of > making a comment (Thanks Elijah!) > * Patch 12 adds messaging to the end of the rebase stating which refs were > updated (Thanks Elijah!) > > > Updates in v4 > ============= > > This version took longer than I'd hoped (I had less time to work on it than > anticipated) but it also has some major updates. These major updates are > direct responses to the significant review this series has received. Thank > you! > > * The update-refs file now stores "ref/before/after" triples (still > separated by lines). This allows us to store the "before" OID of a ref in > addition to the "after" that we will write to that ref at the end of the > rebase. This allows us to do a "force-with-lease" update. The > branch_checked_out() updates should prevent Git from updating those refs > while under the rebase, but older versions and third-party tools don't > have that protection. > * The update-refs file is updated with every update to the todo-list file. > This allows for some advanced changes to the file, including removing, > adding, and duplicating 'update-ref' commands. > * The message at the end of the rebase process now lists which refs were > updated with the update-ref steps. This includes any ref updates that > fail. > * The branch_checked_out() tests now use 'git bisect' and 'git rebase' as > black-boxes instead of testing their internals directly. > > Here are the more minor updates: > > * Dropped an unnecessary stat() call. > * Updated commit messages to include extra details, based on confusion in > last round. > * The HEAD branch no longer appears as a comment line in the initial todo > list. > * The update-refs file is now written using a lockfile. > * Tests now use test_cmp_rev. > * A memory leak ('path' variable) is resolved. > > > Updates in v3 > ============= > > * The branch_checked_out() API was extracted to its own topic and is now > the ds/branch-checked-out branch. This series is now based on that one. > * The for_each_decoration() API was removed, since it became trivial once > it did not take a commit directly. > * The branch_checked_out() tests did not verify the rebase-apply data (for > the apply backend), so that is fixed. > * Instead of using the 'label' command and a final 'update-refs' command in > the todo list, use a new 'update-ref ' command. This command updates the > rebase-merge/update-refs file with the OID of HEAD at these steps. At the > very end of the rebase sequence, those refs are updated to the stored OID > values (assuming that they were not removed by the user, in which case we > notice that the OID is the null OID and we do nothing). > * New tests are added. > * The todo-list comment documentation has some new formatting updates, but > also includes a description of 'update-refs' in this version. > > > Updates in v2 > ============= > > As recommended by the excellent feedback, I have removed the 'exec' commands > in favor of the 'label' commands and a new 'update-refs' command at the very > end. This way, there is only one step that updates all of the refs at the > end instead of updating refs during the rebase. If a user runs 'git rebase > --abort' in the middle, then their refs are still where they need to be. > > Based on some of the discussion, it seemed like one way to do this would be > to have an 'update-ref ' command that would take the place of these 'label' > commands. However, this would require two things that make it a bit awkward: > > 1. We would need to replicate the storage of those positions during the > rebase. 'label' already does this pretty well. I've added the > "for-update-refs/" label to help here. > 2. If we want to close out all of the refs as the rebase is finishing, then > that "step" becomes invisible to the user (and a bit more complicated to > insert). Thus, the 'update-refs' step performs this action. If the user > wants to do things after that step, then they can do so by editing the > TODO list. > > Other updates: > > * The 'keep_decorations' parameter was renamed to 'update_refs'. > * I added tests for --rebase-merges=rebase-cousins to show how these labels > interact with other labels and merge commands. > * I changed the order of the insertion of these update-refs labels to be > before the fixups are rearranged. This fixes a bug where the tip commit > is a fixup! so its decorations are never inspected (and they would be in > the wrong place even if they were). The fixup! commands are properly > inserted between a pick and its following label command. Tests > demonstrate this is correct. > * Numerous style choices are updated based on feedback. > > Thank you for all of the detailed review and ideas in this space. I > appreciate any more ideas that can make this feature as effective as it can > be. > > Thanks, -Stolee > > Derrick Stolee (12): > t2407: test bisect and rebase as black-boxes > t2407: test branches currently using apply backend > branch: consider refs under 'update-refs' > rebase-interactive: update 'merge' description > sequencer: define array with enum values > sequencer: add update-ref command > rebase: add --update-refs option > rebase: update refs from 'update-ref' commands > sequencer: rewrite update-refs as user edits todo list > rebase: add rebase.updateRefs config option > sequencer: ignore HEAD ref under --update-refs > sequencer: notify user of --update-refs activity > > Documentation/config/rebase.txt | 3 + > Documentation/git-rebase.txt | 11 + > branch.c | 13 + > builtin/rebase.c | 10 + > rebase-interactive.c | 15 +- > sequencer.c | 469 +++++++++++++++++++++++++++++++- > sequencer.h | 23 ++ > t/lib-rebase.sh | 15 + > t/t2407-worktree-heads.sh | 103 +++++-- > t/t3404-rebase-interactive.sh | 269 ++++++++++++++++++ > 10 files changed, 887 insertions(+), 44 deletions(-) > > > base-commit: 9bef0b1e6ec371e786c2fba3edcc06ad040a536c > Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1247%2Fderrickstolee%2Frebase-keep-decorations-v4 > Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1247/derrickstolee/rebase-keep-decorations-v4 > Pull-Request: https://github.com/gitgitgadget/git/pull/1247 > > Range-diff vs v3: > > -: ----------- > 1: 9e53a27017a t2407: test bisect and rebase as black-boxes > 1: fbaedc7f1f0 ! 2: 540a3be256f t2407: test branches currently using apply backend > @@ Commit message > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > > ## t/t2407-worktree-heads.sh ## > -@@ t/t2407-worktree-heads.sh: test_expect_success 'refuse to overwrite: worktree in bisect' ' > - grep "cannot force update the branch '\''fake-2'\'' checked out at.*wt-4" err > +@@ t/t2407-worktree-heads.sh: test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in bisect' ' > + grep "cannot force update the branch '\''wt-4'\'' checked out at.*wt-4" err > ' > > --test_expect_success 'refuse to overwrite: worktree in rebase' ' > -+test_expect_success 'refuse to overwrite: worktree in rebase (apply)' ' > -+ test_when_finished rm -rf .git/worktrees/wt-*/rebase-apply && > +-test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase' ' > ++test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (apply)' ' > ++ test_when_finished git -C wt-2 rebase --abort && > + > -+ mkdir -p .git/worktrees/wt-3/rebase-apply && > -+ echo refs/heads/fake-1 >.git/worktrees/wt-3/rebase-apply/head-name && > -+ echo refs/heads/fake-2 >.git/worktrees/wt-3/rebase-apply/onto && > ++ # This will fail part-way through due to a conflict. > ++ test_must_fail git -C wt-2 rebase --apply conflict-2 && > + > -+ test_must_fail git branch -f fake-1 HEAD 2>err && > -+ grep "cannot force update the branch '\''fake-1'\'' checked out at.*wt-3" err > ++ test_must_fail git branch -f wt-2 HEAD 2>err && > ++ grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err > +' > + > -+test_expect_success 'refuse to overwrite: worktree in rebase (merge)' ' > - test_when_finished rm -rf .git/worktrees/wt-*/rebase-merge && > ++test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (merge)' ' > + test_when_finished git -C wt-2 rebase --abort && > > - mkdir -p .git/worktrees/wt-3/rebase-merge && > + # This will fail part-way through due to a conflict. > 2: 2bc647b6fcd ! 3: bf301a054e3 branch: consider refs under 'update-refs' > @@ Commit message > branch_checked_out(). > > The data store is a plaintext file inside the 'rebase-merge' directory > - for that worktree. The file alternates refnames and OIDs. The OIDs will > - be used to store the to-be-written values as the rebase progresses, but > - can be ignored at the moment. > + for that worktree. The file lists refnames followed by two OIDs, each on > + separate lines. The OIDs will be used to store the original values of > + the refs and the to-be-written values as the rebase progresses, but can > + be ignored at the moment. > > Create a new sequencer_get_update_refs_state() method that parses this > file and populates a struct string_list with the ref-OID pairs. We can > @@ Commit message > method. > > We can test that this works without having Git write this file by > - artificially creating one in our test script. > + artificially creating one in our test script, at least until 'git rebase > + --update-refs' is implemented and we can use it directly. > > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > > @@ branch.c: static void prepare_checked_out_branches(void) > free_worktrees(worktrees); > > ## sequencer.c ## > +@@ sequencer.c: static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") > + */ > + static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") > + > ++/* > ++ * The update-refs file stores a list of refs that will be updated at the end > ++ * of the rebase sequence. The 'update-ref <ref>' commands in the todo file > ++ * update the OIDs for the refs in this file, but the refs are not updated > ++ * until the end of the rebase sequence. > ++ * > ++ * rebase_path_update_refs() returns the path to this file for a given > ++ * worktree directory. For the current worktree, pass the_repository->gitdir. > ++ */ > ++static char *rebase_path_update_refs(const char *wt_dir) > ++{ > ++ return xstrfmt("%s/rebase-merge/update-refs", wt_dir); > ++} > ++ > + /* > + * The following files are written by git-rebase just after parsing the > + * command-line. > +@@ sequencer.c: static GIT_PATH_FUNC(rebase_path_no_reschedule_failed_exec, "rebase-merge/no-res > + static GIT_PATH_FUNC(rebase_path_drop_redundant_commits, "rebase-merge/drop_redundant_commits") > + static GIT_PATH_FUNC(rebase_path_keep_redundant_commits, "rebase-merge/keep_redundant_commits") > + > ++/** > ++ * A 'struct update_refs_record' represents a value in the update-refs > ++ * list. We use a string_list to map refs to these (before, after) pairs. > ++ */ > ++struct update_ref_record { > ++ struct object_id before; > ++ struct object_id after; > ++}; > ++ > + static int git_sequencer_config(const char *k, const char *v, void *cb) > + { > + struct replay_opts *opts = cb; > @@ sequencer.c: int sequencer_determine_whence(struct repository *r, enum commit_whence *whence) > > return 0; > @@ sequencer.c: int sequencer_determine_whence(struct repository *r, enum commit_wh > + struct string_list *refs) > +{ > + int result = 0; > -+ struct stat st; > + FILE *fp = NULL; > + struct strbuf ref = STRBUF_INIT; > + struct strbuf hash = STRBUF_INIT; > -+ char *path = xstrfmt("%s/rebase-merge/update-refs", wt_dir); > ++ struct update_ref_record *rec = NULL; > + > -+ if (stat(path, &st)) > -+ goto cleanup; > ++ char *path = rebase_path_update_refs(wt_dir); > + > + fp = fopen(path, "r"); > + if (!fp) > + goto cleanup; > + > + while (strbuf_getline(&ref, fp) != EOF) { > -+ struct object_id oid; > + struct string_list_item *item; > + > ++ CALLOC_ARRAY(rec, 1); > ++ > + if (strbuf_getline(&hash, fp) == EOF || > -+ get_oid_hex(hash.buf, &oid)) { > ++ get_oid_hex(hash.buf, &rec->before)) { > + warning(_("update-refs file at '%s' is invalid"), > + path); > + result = -1; > + goto cleanup; > + } > + > -+ item = string_list_append(refs, ref.buf); > -+ item->util = oiddup(&oid); > ++ if (strbuf_getline(&hash, fp) == EOF || > ++ get_oid_hex(hash.buf, &rec->after)) { > ++ warning(_("update-refs file at '%s' is invalid"), > ++ path); > ++ result = -1; > ++ goto cleanup; > ++ } > ++ > ++ item = string_list_insert(refs, ref.buf); > ++ item->util = rec; > ++ rec = NULL; > + } > + > +cleanup: > + if (fp) > + fclose(fp); > + free(path); > ++ free(rec); > + strbuf_release(&ref); > + strbuf_release(&hash); > + return result; > @@ sequencer.h: void sequencer_post_commit_cleanup(struct repository *r, int verbos > #endif /* SEQUENCER_H */ > > ## t/t2407-worktree-heads.sh ## > -@@ t/t2407-worktree-heads.sh: TEST_PASSES_SANITIZE_LEAK=true > +@@ t/t2407-worktree-heads.sh: test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer > + grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err > + ' > > - test_expect_success 'setup' ' > - test_commit init && > -- git branch -f fake-1 && > -- git branch -f fake-2 && > ++test_expect_success 'refuse to overwrite: worktree in rebase with --update-refs' ' > ++ test_when_finished rm -rf .git/worktrees/wt-3/rebase-merge && > ++ > ++ mkdir -p .git/worktrees/wt-3/rebase-merge && > ++ touch .git/worktrees/wt-3/rebase-merge/interactive && > + > -+ for i in 1 2 3 4 > -+ do > -+ git branch -f fake-$i || return 1 > -+ done && > - > - for i in 1 2 3 4 > - do > -@@ t/t2407-worktree-heads.sh: test_expect_success 'refuse to overwrite: worktree in rebase (merge)' ' > - echo refs/heads/fake-1 >.git/worktrees/wt-3/rebase-merge/head-name && > - echo refs/heads/fake-2 >.git/worktrees/wt-3/rebase-merge/onto && > - > -- test_must_fail git branch -f fake-1 HEAD 2>err && > -- grep "cannot force update the branch '\''fake-1'\'' checked out at.*wt-3" err > + cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && > + refs/heads/fake-3 > + $(git rev-parse HEAD~1) > ++ $(git rev-parse HEAD) > + refs/heads/fake-4 > + $(git rev-parse HEAD) > ++ $(git rev-parse HEAD) > + EOF > + > -+ for i in 1 3 4 > ++ for i in 3 4 > + do > + test_must_fail git branch -f fake-$i HEAD 2>err && > + grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || > + return 1 > + done > - ' > - > ++' > ++ > test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: checked out' ' > + test_must_fail git fetch server +refs/heads/wt-3:refs/heads/wt-3 2>err && > + grep "refusing to fetch into branch '\''refs/heads/wt-3'\''" err && > 3: 669f4abd59e ! 4: dec95681d2b rebase-interactive: update 'merge' description > @@ Commit message > The 'merge' command description for the todo list documentation in an > interactive rebase has multiple lines. The lines other than the first > one start with dots ('.') while the similar multi-line documentation for > - 'fixup' does not. > + 'fixup' does not. This description only appears in the comment text of > + the todo file during an interactive rebase. > > The 'merge' command was documented when interactive rebase was first > ported to C in 145e05ac44b (rebase -i: rewrite append_todo_help() in C, > 4: 6528a50343f = 5: b2c09600918 sequencer: define array with enum values > 5: e95ad41d355 = 6: fa7ecb718cf sequencer: add update-ref command > 6: 918b398d6a2 ! 7: 3ec2cc922f9 rebase: add --update-refs option > @@ Commit message > 'update-ref' commands. Tests are added to ensure that these todo > commands are added in the correct locations. > > - A future change will update the behavior to actually update the refs > - at the end of the rebase sequence. > + This change does _not_ include the actual behavior of tracking the > + updated refs and writing the new ref values at the end of the rebase > + process. That is deferred to a later change. > > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > > @@ sequencer.c: static int skip_unnecessary_picks(struct repository *r, > + item->command = TODO_UPDATE_REF; > + strbuf_addf(ctx->buf, "%s\n", decoration->name); > + > -+ sti = string_list_append(&ctx->refs_to_oids, > ++ sti = string_list_insert(&ctx->refs_to_oids, > + decoration->name); > + sti->util = oiddup(the_hash_algo->null_oid); > + } > @@ t/t2407-worktree-heads.sh: test_expect_success 'refuse to overwrite when in erro > + grep "update-ref refs/heads/allow-update" todo > + ) > +' > ++ > ++# This must be the last test in this file > ++test_expect_success '$EDITOR and friends are unchanged' ' > ++ test_editor_unchanged > ++' > + > test_done > > 7: 72e0481b643 ! 8: fb5f64c5201 rebase: update refs from 'update-ref' commands > @@ Commit message > interactive rebase. > > Teach Git to record the HEAD position when reaching these 'update-ref' > - commands. The ref/OID pair is stored in the > + commands. The ref/before/after triple is stored in the > $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this > file to avoid having other processes updating the refs in that file > while the rebase is in progress. > @@ Commit message > Not only do we update the file when the sequencer reaches these > 'update-ref' commands, we then update the refs themselves at the end of > the rebase sequence. If the rebase is aborted before this final step, > - then the refs are not updated. > + then the refs are not updated. The 'before' value is used to ensure that > + we do not accidentally obliterate a ref that was updated concurrently > + (say, by an older version of Git or a third-party tool). > + > + Now that the 'git rebase --update-refs' command is implemented to write > + to the update-refs file, we can remove the fake construction of the > + update-refs file from a test in t2407-worktree-heads.sh. > > Signed-off-by: Derrick Stolee <derrickstolee@github.com> > > @@ sequencer.c > > #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" > > -@@ sequencer.c: static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") > - */ > - static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") > +@@ sequencer.c: struct update_ref_record { > + struct object_id after; > + }; > > -+/* > -+ * The update-refs file stores a list of refs that will be updated at the end > -+ * of the rebase sequence. The 'update-ref <ref>' commands in the todo file > -+ * update the OIDs for the refs in this file, but the refs are not updated > -+ * until the end of the rebase sequence. > -+ */ > -+static GIT_PATH_FUNC(rebase_path_update_refs, "rebase-merge/update-refs") > -+ > - /* > - * The following files are written by git-rebase just after parsing the > - * command-line. > ++static struct update_ref_record *init_update_ref_record(const char *ref) > ++{ > ++ struct update_ref_record *rec = xmalloc(sizeof(*rec)); > ++ > ++ oidcpy(&rec->before, null_oid()); > ++ oidcpy(&rec->after, null_oid()); > ++ > ++ /* This may fail, but that's fine, we will keep the null OID. */ > ++ read_ref(ref, &rec->before); > ++ > ++ return rec; > ++} > ++ > + static int git_sequencer_config(const char *k, const char *v, void *cb) > + { > + struct replay_opts *opts = cb; > @@ sequencer.c: leave_merge: > return ret; > } > > -static int do_update_ref(struct repository *r, const char *ref_name) > +static int write_update_refs_state(struct string_list *refs_to_oids) > -+{ > + { > + int result = 0; > ++ struct lock_file lock = LOCK_INIT; > + FILE *fp = NULL; > + struct string_list_item *item; > -+ char *path = xstrdup(rebase_path_update_refs()); > ++ char *path; > ++ > ++ if (!refs_to_oids->nr) > ++ return 0; > ++ > ++ path = rebase_path_update_refs(the_repository->gitdir); > + > + if (safe_create_leading_directories(path)) { > + result = error(_("unable to create leading directories of %s"), > @@ sequencer.c: leave_merge: > + goto cleanup; > + } > + > -+ fp = fopen(path, "w"); > ++ if (hold_lock_file_for_update(&lock, path, 0) < 0) { > ++ result = error(_("another 'rebase' process appears to be running; " > ++ "'%s.lock' already exists"), > ++ path); > ++ goto cleanup; > ++ } > ++ > ++ fp = fdopen_lock_file(&lock, "w"); > + if (!fp) { > + result = error_errno(_("could not open '%s' for writing"), path); > ++ rollback_lock_file(&lock); > + goto cleanup; > + } > + > -+ for_each_string_list_item(item, refs_to_oids) > -+ fprintf(fp, "%s\n%s\n", item->string, oid_to_hex(item->util)); > ++ for_each_string_list_item(item, refs_to_oids) { > ++ struct update_ref_record *rec = item->util; > ++ fprintf(fp, "%s\n%s\n%s\n", item->string, > ++ oid_to_hex(&rec->before), oid_to_hex(&rec->after)); > ++ } > ++ > ++ result = commit_lock_file(&lock); > + > +cleanup: > -+ if (fp) > -+ fclose(fp); > ++ free(path); > + return result; > +} > + > +static int do_update_ref(struct repository *r, const char *refname) > - { > ++{ > + struct string_list_item *item; > + struct string_list list = STRING_LIST_INIT_DUP; > -+ int found = 0; > + > + sequencer_get_update_refs_state(r->gitdir, &list); > + > + for_each_string_list_item(item, &list) { > + if (!strcmp(item->string, refname)) { > -+ struct object_id oid; > -+ free(item->util); > -+ found = 1; > -+ > -+ if (!read_ref("HEAD", &oid)) { > -+ item->util = oiddup(&oid); > -+ break; > -+ } > ++ struct update_ref_record *rec = item->util; > ++ read_ref("HEAD", &rec->after); > ++ break; > + } > + } > + > -+ if (!found) { > -+ struct object_id oid; > -+ item = string_list_append(&list, refname); > -+ > -+ if (!read_ref("HEAD", &oid)) > -+ item->util = oiddup(&oid); > -+ else > -+ item->util = oiddup(the_hash_algo->null_oid); > -+ } > -+ > + write_update_refs_state(&list); > + string_list_clear(&list, 1); > return 0; > @@ sequencer.c: leave_merge: > + sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); > + > + for_each_string_list_item(item, &refs_to_oids) { > -+ struct object_id *oid_to = item->util; > -+ struct object_id oid_from; > ++ struct update_ref_record *rec = item->util; > + > -+ if (oideq(oid_to, the_hash_algo->null_oid)) { > ++ if (oideq(&rec->after, the_hash_algo->null_oid)) { > + /* > + * Ref was not updated. User may have deleted the > + * 'update-ref' step. > @@ sequencer.c: leave_merge: > + continue; > + } > + > -+ if (read_ref(item->string, &oid_from)) { > -+ /* > -+ * The ref does not exist. The user probably > -+ * inserted a new 'update-ref' step with a new > -+ * branch name. > -+ */ > -+ oidcpy(&oid_from, the_hash_algo->null_oid); > -+ } > -+ > + res |= refs_update_ref(refs, "rewritten during rebase", > -+ item->string, > -+ oid_to, &oid_from, > -+ 0, UPDATE_REFS_MSG_ON_ERR); > ++ item->string, > ++ &rec->after, &rec->before, > ++ 0, UPDATE_REFS_MSG_ON_ERR); > + } > + > + string_list_clear(&refs_to_oids, 1); > @@ sequencer.c: cleanup_head_ref: > /* > * Sequence of picks finished successfully; cleanup by > * removing the .git/sequencer directory > +@@ sequencer.c: static int add_decorations_to_list(const struct commit *commit, > + > + sti = string_list_insert(&ctx->refs_to_oids, > + decoration->name); > +- sti->util = oiddup(the_hash_algo->null_oid); > ++ sti->util = init_update_ref_record(decoration->name); > + } > + > + item->offset_in_buf = base_offset; > @@ sequencer.c: static int todo_list_add_update_ref_commands(struct todo_list *todo_list) > } > } > @@ sequencer.c: static int todo_list_add_update_ref_commands(struct todo_list *todo > free(todo_list->items); > todo_list->items = ctx.items; > > + ## t/t2407-worktree-heads.sh ## > +@@ t/t2407-worktree-heads.sh: test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer > + grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err > + ' > + > +-test_expect_success 'refuse to overwrite: worktree in rebase with --update-refs' ' > +- test_when_finished rm -rf .git/worktrees/wt-3/rebase-merge && > +- > +- mkdir -p .git/worktrees/wt-3/rebase-merge && > +- touch .git/worktrees/wt-3/rebase-merge/interactive && > ++test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase with --update-refs' ' > ++ test_when_finished git -C wt-3 rebase --abort && > + > +- cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && > +- refs/heads/fake-3 > +- $(git rev-parse HEAD~1) > +- $(git rev-parse HEAD) > +- refs/heads/fake-4 > +- $(git rev-parse HEAD) > +- $(git rev-parse HEAD) > +- EOF > ++ git branch -f can-be-updated wt-3 && > ++ test_must_fail git -C wt-3 rebase --update-refs conflict-3 && > + > + for i in 3 4 > + do > +- test_must_fail git branch -f fake-$i HEAD 2>err && > +- grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || > ++ test_must_fail git branch -f can-be-updated HEAD 2>err && > ++ grep "cannot force update the branch '\''can-be-updated'\'' checked out at.*wt-3" err || > + return 1 > + done > + ' > + > ## t/t3404-rebase-interactive.sh ## > @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds commands with --rebase-merges' ' > ) > ' > > -+compare_two_refs () { > -+ git rev-parse $1 >expect && > -+ git rev-parse $2 >actual && > -+ test_cmp expect actual > -+} > -+ > +test_expect_success '--update-refs updates refs correctly' ' > + git checkout -B update-refs no-conflict-branch && > + git branch -f base HEAD~4 && > @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs adds commands > + > + git rebase -i --autosquash --update-refs primary && > + > -+ compare_two_refs HEAD~3 refs/heads/first && > -+ compare_two_refs HEAD~3 refs/heads/second && > -+ compare_two_refs HEAD~1 refs/heads/third && > -+ compare_two_refs HEAD refs/heads/no-conflict-branch > ++ test_cmp_rev HEAD~3 refs/heads/first && > ++ test_cmp_rev HEAD~3 refs/heads/second && > ++ test_cmp_rev HEAD~1 refs/heads/third && > ++ test_cmp_rev HEAD refs/heads/no-conflict-branch > +' > + > # This must be the last test in this file > -: ----------- > 9: 29c7c76805a sequencer: rewrite update-refs as user edits todo list > 8: d2cfdbfc431 = 10: c0022d07579 rebase: add rebase.updateRefs config option > -: ----------- > 11: d53b4ff2cee sequencer: ignore HEAD ref under --update-refs > -: ----------- > 12: d5cd4b49e46 sequencer: notify user of --update-refs activity > ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v5 00/12] rebase: update branches in multi-part topic 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget ` (13 preceding siblings ...) 2022-07-15 15:41 ` Phillip Wood @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 01/12] t2407: test bisect and rebase as black-boxes Derrick Stolee via GitGitGadget ` (13 more replies) 14 siblings, 14 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee This series is based on ds/branch-checked-out. This is a feature I've wanted for quite a while. When working on the sparse index topic, I created a long RFC that actually broke into three topics for full review upstream. These topics were sequential, so any feedback on an earlier one required updates to the later ones. I would work on the full feature and use interactive rebase to update the full list of commits. However, I would need to update the branches pointing to those sub-topics. This series adds a new --update-refs option to 'git rebase' (along with a rebase.updateRefs config option) that adds 'update-ref' commands into the TODO list. This is powered by the commit decoration machinery. As an example, here is my in-progress bundle URI RFC split into subtopics as they appear during the TODO list of a git rebase -i --update-refs: pick 2d966282ff3 docs: document bundle URI standard pick 31396e9171a remote-curl: add 'get' capability pick 54c6ab70f67 bundle-uri: create basic file-copy logic pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// pick 6adaf842684 fetch: add --bundle-uri option pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration update-ref refs/heads/bundle-redo/fetch pick 1e3f6546632 clone: add --bundle-uri option pick 9e4a6fe9b68 clone: --bundle-uri cannot be combined with --depth update-ref refs/heads/bundle-redo/clone pick 5451cb6599c bundle-uri: create bundle_list struct and helpers pick 3029c3aca15 bundle-uri: create base key-value pair parsing pick a8b2de79ce8 bundle-uri: create "key=value" line parsing pick 92625a47673 bundle-uri: unit test "key=value" parsing pick a8616af4dc2 bundle-uri: limit recursion depth for bundle lists pick 9d6809a8d53 bundle-uri: parse bundle list in config format pick 287a732b54c bundle-uri: fetch a list of bundles update-ref refs/heads/bundle-redo/list pick b09f8226185 protocol v2: add server-side "bundle-uri" skeleton pick 520204dcd1c bundle-uri client: add minimal NOOP client pick 62e8b457b48 bundle-uri client: add "git ls-remote-bundle-uri" pick 00eae925043 bundle-uri: serve URI advertisement from bundle.* config pick 4277440a250 bundle-uri client: add boolean transfer.bundleURI setting pick caf4599a81d bundle-uri: allow relative URLs in bundle lists pick df255000b7e bundle-uri: download bundles from an advertised list pick d71beabf199 clone: unbundle the advertised bundles pick c9578391976 t5601: basic bundle URI tests # Ref refs/heads/bundle-redo/rfc-3 checked out at '/home/stolee/_git/git-bundles' update-ref refs/heads/bundle-redo/advertise Here is an outline of the series: * Patch 1 updates some tests for branch_checked_out() to use 'git bisect' and 'git rebase' as black-boxes instead of manually editing files inside $GIT_DIR. (Thanks, Junio!) * Patch 2 updates some tests for branch_checked_out() for the 'apply' backend. * Patch 3 updates branch_checked_out() to parse the rebase-merge/update-refs file to block concurrent ref updates and checkouts on branches selected by --update-refs. * Patch 4 updates the todo list documentation to remove some unnecessary dots in the 'merge' command. This makes it consistent with the 'fixup' command before we document the 'update-ref' command. * Patch 5 updates the definition of todo_command_info to use enum values as array indices. * Patches 6-8 implement the --update-refs logic itself. * Patch 9 specifically updates the update-refs file every time the user edits the todo-list (Thanks Phillip!) * Patch 10 adds the rebase.updateRefs config option similar to rebase.autoSquash. * Patch 11 ignores the HEAD ref when creating the todo list instead of making a comment (Thanks Elijah!) * Patch 12 adds messaging to the end of the rebase stating which refs were updated (Thanks Elijah!) During review, we have identified some areas that would be good for #leftoverbits: * Warn the user when they add an 'update-ref ' command but is checked out in another worktree. * The checks in patch 9 are quadratic. They could be sped up using hashtables. * Consider whether we should include an 'update-ref ' command for the HEAD ref, so that all refs are updated in the same way. This might help confused users. * The error message for failed ref updates could include information on the commit IDs that would have been used. This can help the user fix the situation by updating the refs manually. * Modify the --update-refs option from a boolean to an optionally-string-parameter that specifies refspecs for the 'update-ref' commands. Updates in v5 ============= * Rename 'wt_dir' to 'wt_git_dir' for clarity. * The documented behavior around 'fixup!' and 'squash!' commits was incorrect, so update the commit message, documentation, and test to demonstrate the actual behavior. * Use CALLOC_ARRAY() to be more idiomatic. * Be much more careful about propagating errors. * Commit message typo: "We an" to "We can" * Remove unnecessary null OID check when writing refs, since those would already be removed by a previous step. Updates in v4 ============= This version took longer than I'd hoped (I had less time to work on it than anticipated) but it also has some major updates. These major updates are direct responses to the significant review this series has received. Thank you! * The update-refs file now stores "ref/before/after" triples (still separated by lines). This allows us to store the "before" OID of a ref in addition to the "after" that we will write to that ref at the end of the rebase. This allows us to do a "force-with-lease" update. The branch_checked_out() updates should prevent Git from updating those refs while under the rebase, but older versions and third-party tools don't have that protection. * The update-refs file is updated with every update to the todo-list file. This allows for some advanced changes to the file, including removing, adding, and duplicating 'update-ref' commands. * The message at the end of the rebase process now lists which refs were updated with the update-ref steps. This includes any ref updates that fail. * The branch_checked_out() tests now use 'git bisect' and 'git rebase' as black-boxes instead of testing their internals directly. Here are the more minor updates: * Dropped an unnecessary stat() call. * Updated commit messages to include extra details, based on confusion in last round. * The HEAD branch no longer appears as a comment line in the initial todo list. * The update-refs file is now written using a lockfile. * Tests now use test_cmp_rev. * A memory leak ('path' variable) is resolved. Updates in v3 ============= * The branch_checked_out() API was extracted to its own topic and is now the ds/branch-checked-out branch. This series is now based on that one. * The for_each_decoration() API was removed, since it became trivial once it did not take a commit directly. * The branch_checked_out() tests did not verify the rebase-apply data (for the apply backend), so that is fixed. * Instead of using the 'label' command and a final 'update-refs' command in the todo list, use a new 'update-ref ' command. This command updates the rebase-merge/update-refs file with the OID of HEAD at these steps. At the very end of the rebase sequence, those refs are updated to the stored OID values (assuming that they were not removed by the user, in which case we notice that the OID is the null OID and we do nothing). * New tests are added. * The todo-list comment documentation has some new formatting updates, but also includes a description of 'update-refs' in this version. Updates in v2 ============= As recommended by the excellent feedback, I have removed the 'exec' commands in favor of the 'label' commands and a new 'update-refs' command at the very end. This way, there is only one step that updates all of the refs at the end instead of updating refs during the rebase. If a user runs 'git rebase --abort' in the middle, then their refs are still where they need to be. Based on some of the discussion, it seemed like one way to do this would be to have an 'update-ref ' command that would take the place of these 'label' commands. However, this would require two things that make it a bit awkward: 1. We would need to replicate the storage of those positions during the rebase. 'label' already does this pretty well. I've added the "for-update-refs/" label to help here. 2. If we want to close out all of the refs as the rebase is finishing, then that "step" becomes invisible to the user (and a bit more complicated to insert). Thus, the 'update-refs' step performs this action. If the user wants to do things after that step, then they can do so by editing the TODO list. Other updates: * The 'keep_decorations' parameter was renamed to 'update_refs'. * I added tests for --rebase-merges=rebase-cousins to show how these labels interact with other labels and merge commands. * I changed the order of the insertion of these update-refs labels to be before the fixups are rearranged. This fixes a bug where the tip commit is a fixup! so its decorations are never inspected (and they would be in the wrong place even if they were). The fixup! commands are properly inserted between a pick and its following label command. Tests demonstrate this is correct. * Numerous style choices are updated based on feedback. Thank you for all of the detailed review and ideas in this space. I appreciate any more ideas that can make this feature as effective as it can be. Thanks, -Stolee Derrick Stolee (12): t2407: test bisect and rebase as black-boxes t2407: test branches currently using apply backend branch: consider refs under 'update-refs' rebase-interactive: update 'merge' description sequencer: define array with enum values sequencer: add update-ref command rebase: add --update-refs option rebase: update refs from 'update-ref' commands sequencer: rewrite update-refs as user edits todo list rebase: add rebase.updateRefs config option sequencer: ignore HEAD ref under --update-refs sequencer: notify user of --update-refs activity Documentation/config/rebase.txt | 3 + Documentation/git-rebase.txt | 10 + branch.c | 13 + builtin/rebase.c | 10 + rebase-interactive.c | 15 +- sequencer.c | 474 +++++++++++++++++++++++++++++++- sequencer.h | 23 ++ t/lib-rebase.sh | 15 + t/t2407-worktree-heads.sh | 103 +++++-- t/t3404-rebase-interactive.sh | 273 ++++++++++++++++++ 10 files changed, 895 insertions(+), 44 deletions(-) base-commit: 9bef0b1e6ec371e786c2fba3edcc06ad040a536c Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1247%2Fderrickstolee%2Frebase-keep-decorations-v5 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1247/derrickstolee/rebase-keep-decorations-v5 Pull-Request: https://github.com/gitgitgadget/git/pull/1247 Range-diff vs v4: 1: 9e53a27017a = 1: 9e53a27017a t2407: test bisect and rebase as black-boxes 2: 540a3be256f = 2: 540a3be256f t2407: test branches currently using apply backend 3: bf301a054e3 ! 3: 1089a0edb73 branch: consider refs under 'update-refs' @@ sequencer.c: static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash- + * rebase_path_update_refs() returns the path to this file for a given + * worktree directory. For the current worktree, pass the_repository->gitdir. + */ -+static char *rebase_path_update_refs(const char *wt_dir) ++static char *rebase_path_update_refs(const char *wt_git_dir) +{ -+ return xstrfmt("%s/rebase-merge/update-refs", wt_dir); ++ return xstrfmt("%s/rebase-merge/update-refs", wt_git_dir); +} + /* 4: dec95681d2b = 4: d1cce4f06aa rebase-interactive: update 'merge' description 5: b2c09600918 = 5: 4c086d477f0 sequencer: define array with enum values 6: fa7ecb718cf = 6: 7b3d6601960 sequencer: add update-ref command 7: 3ec2cc922f9 ! 7: 7efb55e4f14 rebase: add --update-refs option @@ Commit message --fixup commit at the tip of the feature to apply correctly to the sub branch, even if it is fixing up the most-recent commit in that part. - One potential problem here is that refs decorating commits that are - already marked as "fixup!" or "squash!" will not be included in this - list. Generally, the reordering of the "fixup!" and "squash!" is likely - to change the relative order of these refs, so it is not recommended. - The workflow here is intended to allow these kinds of commits at the tip - of the rebased branch while the other sub branches come along for the - ride without intervention. - This change update the documentation and builtin to accept the --update-refs option as well as updating the todo file with the 'update-ref' commands. Tests are added to ensure that these todo @@ Documentation/git-rebase.txt: provided. Otherwise an explicit `--no-reschedule-f +--no-update-refs:: + Automatically force-update any branches that point to commits that + are being rebased. Any branches that are checked out in a worktree -+ or point to a `squash! ...` or `fixup! ...` commit are not updated -+ in this way. ++ are not updated in this way. + INCOMPATIBLE OPTIONS -------------------- @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix N_("move commits that begin with " "squash!/fixup! under -i")), + OPT_BOOL(0, "update-refs", &options.update_refs, -+ N_("update local refs that point to commits " ++ N_("update branches that point to commits " + "that are being rebased")), { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"), N_("GPG-sign commits"), @@ t/t3404-rebase-interactive.sh: test_expect_success 'ORIG_HEAD is updated correct + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git commit --allow-empty --fixup=third && ++ git branch -f is-not-reordered && ++ git commit --allow-empty --fixup=HEAD~4 && + git branch -f shared-tip && + ( + set_cat_todo_editor && + + cat >expect <<-EOF && + pick $(git log -1 --format=%h J) J ++ fixup $(git log -1 --format=%h update-refs) fixup! J # empty + update-ref refs/heads/second + update-ref refs/heads/first + pick $(git log -1 --format=%h K) K + pick $(git log -1 --format=%h L) L -+ fixup $(git log -1 --format=%h update-refs) fixup! L # empty ++ fixup $(git log -1 --format=%h is-not-reordered) fixup! L # empty + update-ref refs/heads/third + pick $(git log -1 --format=%h M) M + update-ref refs/heads/no-conflict-branch ++ update-ref refs/heads/is-not-reordered + update-ref refs/heads/shared-tip + EOF + 8: fb5f64c5201 ! 8: e7a91bdffbd rebase: update refs from 'update-ref' commands @@ sequencer.c: struct update_ref_record { +static struct update_ref_record *init_update_ref_record(const char *ref) +{ -+ struct update_ref_record *rec = xmalloc(sizeof(*rec)); ++ struct update_ref_record *rec; ++ ++ CALLOC_ARRAY(rec, 1); + + oidcpy(&rec->before, null_oid()); + oidcpy(&rec->after, null_oid()); @@ sequencer.c: leave_merge: -static int do_update_ref(struct repository *r, const char *ref_name) +static int write_update_refs_state(struct string_list *refs_to_oids) - { ++{ + int result = 0; + struct lock_file lock = LOCK_INIT; + FILE *fp = NULL; @@ sequencer.c: leave_merge: +} + +static int do_update_ref(struct repository *r, const char *refname) -+{ + { + struct string_list_item *item; + struct string_list list = STRING_LIST_INIT_DUP; + -+ sequencer_get_update_refs_state(r->gitdir, &list); ++ if (sequencer_get_update_refs_state(r->gitdir, &list)) ++ return -1; + + for_each_string_list_item(item, &list) { + if (!strcmp(item->string, refname)) { + struct update_ref_record *rec = item->util; -+ read_ref("HEAD", &rec->after); ++ if (read_ref("HEAD", &rec->after)) ++ return -1; + break; + } + } @@ sequencer.c: leave_merge: + struct string_list refs_to_oids = STRING_LIST_INIT_DUP; + struct ref_store *refs = get_main_ref_store(r); + -+ sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); ++ if ((res = sequencer_get_update_refs_state(r->gitdir, &refs_to_oids))) ++ return res; + + for_each_string_list_item(item, &refs_to_oids) { + struct update_ref_record *rec = item->util; + -+ if (oideq(&rec->after, the_hash_algo->null_oid)) { -+ /* -+ * Ref was not updated. User may have deleted the -+ * 'update-ref' step. -+ */ -+ continue; -+ } -+ + res |= refs_update_ref(refs, "rewritten during rebase", + item->string, + &rec->after, &rec->before, @@ sequencer.c: leave_merge: { int i = todo_list->current; @@ sequencer.c: cleanup_head_ref: + + strbuf_release(&buf); strbuf_release(&head_ref); ++ ++ if (do_update_refs(r)) ++ return -1; } -+ do_update_refs(r); -+ /* - * Sequence of picks finished successfully; cleanup by - * removing the .git/sequencer directory @@ sequencer.c: static int add_decorations_to_list(const struct commit *commit, sti = string_list_insert(&ctx->refs_to_oids, @@ sequencer.c: static int add_decorations_to_list(const struct commit *commit, } item->offset_in_buf = base_offset; +@@ sequencer.c: static int add_decorations_to_list(const struct commit *commit, + */ + static int todo_list_add_update_ref_commands(struct todo_list *todo_list) + { +- int i; ++ int i, res; + static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP; @@ sequencer.c: static int todo_list_add_update_ref_commands(struct todo_list *todo_list) } } -+ write_update_refs_state(&ctx.refs_to_oids); ++ res = write_update_refs_state(&ctx.refs_to_oids); + string_list_clear(&ctx.refs_to_oids, 1); ++ ++ if (res) { ++ /* we failed, so clean up the new list. */ ++ free(ctx.items); ++ return res; ++ } ++ free(todo_list->items); todo_list->items = ctx.items; + todo_list->nr = ctx.items_nr; ## t/t2407-worktree-heads.sh ## @@ t/t2407-worktree-heads.sh: test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer 9: 29c7c76805a ! 9: 95e2bbcedb1 sequencer: rewrite update-refs as user edits todo list @@ Commit message We can test that this works by rewriting the todo-list several times in the course of a rebase. Check that each ref is locked or unlocked for - updates after each todo-list update. We an also verify that the ref + updates after each todo-list update. We can also verify that the ref update fails if a concurrent process updates one of the refs after the rebase process records the "locked" ref location. 10: c0022d07579 ! 10: a73b02568f3 rebase: add rebase.updateRefs config option @@ Documentation/config/rebase.txt: rebase.autoStash:: ## Documentation/git-rebase.txt ## @@ Documentation/git-rebase.txt: start would be overridden by the presence of + Automatically force-update any branches that point to commits that are being rebased. Any branches that are checked out in a worktree - or point to a `squash! ...` or `fixup! ...` commit are not updated - in this way. + are not updated in this way. ++ +If the configuration variable `rebase.updateRefs` is set, then this option +can be used to override and disable this setting. 11: d53b4ff2cee = 11: 2a6577974c7 sequencer: ignore HEAD ref under --update-refs 12: d5cd4b49e46 ! 12: ec080ce1e90 sequencer: notify user of --update-refs activity @@ sequencer.c: static int do_update_ref(struct repository *r, const char *refname) + struct strbuf update_msg = STRBUF_INIT; + struct strbuf error_msg = STRBUF_INIT; - sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); + if ((res = sequencer_get_update_refs_state(r->gitdir, &refs_to_oids))) + return res; for_each_string_list_item(item, &refs_to_oids) { struct update_ref_record *rec = item->util; + int loop_res; - if (oideq(&rec->after, the_hash_algo->null_oid)) { - /* -@@ sequencer.c: static int do_update_refs(struct repository *r) - continue; - } - - res |= refs_update_ref(refs, "rewritten during rebase", - item->string, - &rec->after, &rec->before, @@ sequencer.c: static int do_update_refs(struct repository *r) } @@ sequencer.c: cleanup_head_ref: + strbuf_release(&buf); strbuf_release(&head_ref); - } -- do_update_refs(r); -+ do_update_refs(r, opts->quiet); +- if (do_update_refs(r)) ++ if (do_update_refs(r, opts->quiet)) + return -1; + } - /* - * Sequence of picks finished successfully; cleanup by ## t/t3404-rebase-interactive.sh ## @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs updates refs correctly' ' @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs updates refs c test_expect_success 'respect user edits to update-ref steps' ' @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs: check failed ref update' ' + # the lock in the update-refs file. git rev-parse third >.git/refs/heads/second && - git rebase --continue 2>err && +- git rebase --continue 2>err && - grep "update_ref failed for ref '\''refs/heads/second'\''" err ++ test_must_fail git rebase --continue 2>err && + grep "update_ref failed for ref '\''refs/heads/second'\''" err && + + cat >expect <<-\EOF && -- gitgitgadget ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v5 01/12] t2407: test bisect and rebase as black-boxes 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 02/12] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget ` (12 subsequent siblings) 13 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The tests added by d2ba271aad0 (branch: check for bisects and rebases, 2022-06-14) modified hidden state to verify the branch_checked_out() helper. While this indeed checks that the method implementation is _as designed_, it doesn't show that it is _correct_. Specifically, if 'git bisect' or 'git rebase' change their back-end for preserving refs, then these tests do not demonstrate that drift as a bug in branch_checked_out(). Modify the tests in t2407 to actually rely on a paused bisect or rebase. This requires adding the !SANITIZE_LEAK prereq for tests using those builtins. The logic is still tested for leaks in the final test which does set up that back-end directly for an error state that should not be possible using Git commands. Reported-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- t/t2407-worktree-heads.sh | 57 +++++++++++++++++++++------------------ 1 file changed, 31 insertions(+), 26 deletions(-) diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index b6be42f74a2..100ab286d5c 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -7,13 +7,18 @@ TEST_PASSES_SANITIZE_LEAK=true test_expect_success 'setup' ' test_commit init && - git branch -f fake-1 && - git branch -f fake-2 && for i in 1 2 3 4 do + git checkout -b conflict-$i && + echo "not I" >$i.t && + git add $i.t && + git commit -m "will conflict" && + + git checkout - && test_commit $i && git branch wt-$i && + git branch fake-$i && git worktree add wt-$i wt-$i || return 1 done && @@ -44,26 +49,26 @@ test_expect_success 'refuse to overwrite: checked out in worktree' ' done ' -test_expect_success 'refuse to overwrite: worktree in bisect' ' - test_when_finished rm -rf .git/worktrees/wt-*/BISECT_* && +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in bisect' ' + test_when_finished git -C wt-4 bisect reset && - touch .git/worktrees/wt-4/BISECT_LOG && - echo refs/heads/fake-2 >.git/worktrees/wt-4/BISECT_START && + # Set up a bisect so HEAD no longer points to wt-4. + git -C wt-4 bisect start && + git -C wt-4 bisect bad wt-4 && + git -C wt-4 bisect good wt-1 && - test_must_fail git branch -f fake-2 HEAD 2>err && - grep "cannot force update the branch '\''fake-2'\'' checked out at.*wt-4" err + test_must_fail git branch -f wt-4 HEAD 2>err && + grep "cannot force update the branch '\''wt-4'\'' checked out at.*wt-4" err ' -test_expect_success 'refuse to overwrite: worktree in rebase' ' - test_when_finished rm -rf .git/worktrees/wt-*/rebase-merge && +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase' ' + test_when_finished git -C wt-2 rebase --abort && - mkdir -p .git/worktrees/wt-3/rebase-merge && - touch .git/worktrees/wt-3/rebase-merge/interactive && - echo refs/heads/fake-1 >.git/worktrees/wt-3/rebase-merge/head-name && - echo refs/heads/fake-2 >.git/worktrees/wt-3/rebase-merge/onto && + # This will fail part-way through due to a conflict. + test_must_fail git -C wt-2 rebase conflict-2 && - test_must_fail git branch -f fake-1 HEAD 2>err && - grep "cannot force update the branch '\''fake-1'\'' checked out at.*wt-3" err + test_must_fail git branch -f wt-2 HEAD 2>err && + grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err ' test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: checked out' ' @@ -77,24 +82,24 @@ test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: checked out' ' ' test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: worktree in bisect' ' - test_when_finished rm -rf .git/worktrees/wt-*/BISECT_* && + test_when_finished git -C wt-4 bisect reset && - touch .git/worktrees/wt-4/BISECT_LOG && - echo refs/heads/fake-2 >.git/worktrees/wt-4/BISECT_START && + # Set up a bisect so HEAD no longer points to wt-4. + git -C wt-4 bisect start && + git -C wt-4 bisect bad wt-4 && + git -C wt-4 bisect good wt-1 && - test_must_fail git fetch server +refs/heads/fake-2:refs/heads/fake-2 2>err && + test_must_fail git fetch server +refs/heads/wt-4:refs/heads/wt-4 2>err && grep "refusing to fetch into branch" err ' test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: worktree in rebase' ' - test_when_finished rm -rf .git/worktrees/wt-*/rebase-merge && + test_when_finished git -C wt-3 rebase --abort && - mkdir -p .git/worktrees/wt-4/rebase-merge && - touch .git/worktrees/wt-4/rebase-merge/interactive && - echo refs/heads/fake-1 >.git/worktrees/wt-4/rebase-merge/head-name && - echo refs/heads/fake-2 >.git/worktrees/wt-4/rebase-merge/onto && + # This will fail part-way through due to a conflict. + test_must_fail git -C wt-3 rebase conflict-3 && - test_must_fail git fetch server +refs/heads/fake-1:refs/heads/fake-1 2>err && + test_must_fail git fetch server +refs/heads/wt-3:refs/heads/wt-3 2>err && grep "refusing to fetch into branch" err ' -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v5 02/12] t2407: test branches currently using apply backend 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 01/12] t2407: test bisect and rebase as black-boxes Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 03/12] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget ` (11 subsequent siblings) 13 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The tests in t2407 that verify the branch_checked_out() helper in the case of bisects and rebases were added by 9347303db89 (branch: check for bisects and rebases, 2022-06-08). However, that commit failed to check for rebases that are using the 'apply' backend. Add a test that checks the apply backend. The implementation was already correct here, but it is good to have regression tests before modifying the implementation further. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- t/t2407-worktree-heads.sh | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index 100ab286d5c..a67ce5fb003 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -61,7 +61,17 @@ test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in bisect' ' grep "cannot force update the branch '\''wt-4'\'' checked out at.*wt-4" err ' -test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase' ' +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (apply)' ' + test_when_finished git -C wt-2 rebase --abort && + + # This will fail part-way through due to a conflict. + test_must_fail git -C wt-2 rebase --apply conflict-2 && + + test_must_fail git branch -f wt-2 HEAD 2>err && + grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err +' + +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (merge)' ' test_when_finished git -C wt-2 rebase --abort && # This will fail part-way through due to a conflict. -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v5 03/12] branch: consider refs under 'update-refs' 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 01/12] t2407: test bisect and rebase as black-boxes Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 02/12] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 04/12] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget ` (10 subsequent siblings) 13 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The branch_checked_out() helper helps commands like 'git branch' and 'git fetch' from overwriting refs that are currently checked out in other worktrees. A future update to 'git rebase' will introduce a new '--update-refs' option which will update the local refs that point to commits that are being rebased. To avoid collisions as the rebase completes, we want to make the future data store for these refs to be considered by branch_checked_out(). The data store is a plaintext file inside the 'rebase-merge' directory for that worktree. The file lists refnames followed by two OIDs, each on separate lines. The OIDs will be used to store the original values of the refs and the to-be-written values as the rebase progresses, but can be ignored at the moment. Create a new sequencer_get_update_refs_state() method that parses this file and populates a struct string_list with the ref-OID pairs. We can then use this list to add to the current_checked_out_branches strmap used by branch_checked_out(). To properly navigate to the rebase directory for a given worktree, extract the static strbuf_worktree_gitdir() method to a public API method. We can test that this works without having Git write this file by artificially creating one in our test script, at least until 'git rebase --update-refs' is implemented and we can use it directly. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- branch.c | 13 +++++++ sequencer.c | 74 +++++++++++++++++++++++++++++++++++++++ sequencer.h | 9 +++++ t/t2407-worktree-heads.sh | 23 ++++++++++++ 4 files changed, 119 insertions(+) diff --git a/branch.c b/branch.c index 526e8237673..f252c4eef6c 100644 --- a/branch.c +++ b/branch.c @@ -365,6 +365,7 @@ static void prepare_checked_out_branches(void) char *old; struct wt_status_state state = { 0 }; struct worktree *wt = worktrees[i++]; + struct string_list update_refs = STRING_LIST_INIT_DUP; if (wt->is_bare) continue; @@ -400,6 +401,18 @@ static void prepare_checked_out_branches(void) strbuf_release(&ref); } wt_status_state_free_buffers(&state); + + if (!sequencer_get_update_refs_state(get_worktree_git_dir(wt), + &update_refs)) { + struct string_list_item *item; + for_each_string_list_item(item, &update_refs) { + old = strmap_put(¤t_checked_out_branches, + item->string, + xstrdup(wt->path)); + free(old); + } + string_list_clear(&update_refs, 1); + } } free_worktrees(worktrees); diff --git a/sequencer.c b/sequencer.c index 8c3ed3532ac..7d131d06cc3 100644 --- a/sequencer.c +++ b/sequencer.c @@ -147,6 +147,20 @@ static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash-onto") */ static GIT_PATH_FUNC(rebase_path_refs_to_delete, "rebase-merge/refs-to-delete") +/* + * The update-refs file stores a list of refs that will be updated at the end + * of the rebase sequence. The 'update-ref <ref>' commands in the todo file + * update the OIDs for the refs in this file, but the refs are not updated + * until the end of the rebase sequence. + * + * rebase_path_update_refs() returns the path to this file for a given + * worktree directory. For the current worktree, pass the_repository->gitdir. + */ +static char *rebase_path_update_refs(const char *wt_git_dir) +{ + return xstrfmt("%s/rebase-merge/update-refs", wt_git_dir); +} + /* * The following files are written by git-rebase just after parsing the * command-line. @@ -169,6 +183,15 @@ static GIT_PATH_FUNC(rebase_path_no_reschedule_failed_exec, "rebase-merge/no-res static GIT_PATH_FUNC(rebase_path_drop_redundant_commits, "rebase-merge/drop_redundant_commits") static GIT_PATH_FUNC(rebase_path_keep_redundant_commits, "rebase-merge/keep_redundant_commits") +/** + * A 'struct update_refs_record' represents a value in the update-refs + * list. We use a string_list to map refs to these (before, after) pairs. + */ +struct update_ref_record { + struct object_id before; + struct object_id after; +}; + static int git_sequencer_config(const char *k, const char *v, void *cb) { struct replay_opts *opts = cb; @@ -5901,3 +5924,54 @@ int sequencer_determine_whence(struct repository *r, enum commit_whence *whence) return 0; } + +int sequencer_get_update_refs_state(const char *wt_dir, + struct string_list *refs) +{ + int result = 0; + FILE *fp = NULL; + struct strbuf ref = STRBUF_INIT; + struct strbuf hash = STRBUF_INIT; + struct update_ref_record *rec = NULL; + + char *path = rebase_path_update_refs(wt_dir); + + fp = fopen(path, "r"); + if (!fp) + goto cleanup; + + while (strbuf_getline(&ref, fp) != EOF) { + struct string_list_item *item; + + CALLOC_ARRAY(rec, 1); + + if (strbuf_getline(&hash, fp) == EOF || + get_oid_hex(hash.buf, &rec->before)) { + warning(_("update-refs file at '%s' is invalid"), + path); + result = -1; + goto cleanup; + } + + if (strbuf_getline(&hash, fp) == EOF || + get_oid_hex(hash.buf, &rec->after)) { + warning(_("update-refs file at '%s' is invalid"), + path); + result = -1; + goto cleanup; + } + + item = string_list_insert(refs, ref.buf); + item->util = rec; + rec = NULL; + } + +cleanup: + if (fp) + fclose(fp); + free(path); + free(rec); + strbuf_release(&ref); + strbuf_release(&hash); + return result; +} diff --git a/sequencer.h b/sequencer.h index da64473636b..3ae541bb145 100644 --- a/sequencer.h +++ b/sequencer.h @@ -232,4 +232,13 @@ void sequencer_post_commit_cleanup(struct repository *r, int verbose); int sequencer_get_last_command(struct repository* r, enum replay_action *action); int sequencer_determine_whence(struct repository *r, enum commit_whence *whence); + +/** + * Append the set of ref-OID pairs that are currently stored for the 'git + * rebase --update-refs' feature if such a rebase is currently happening. + * + * Localized to a worktree's git dir. + */ +int sequencer_get_update_refs_state(const char *wt_dir, struct string_list *refs); + #endif /* SEQUENCER_H */ diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index a67ce5fb003..97f5c87f8c8 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -81,6 +81,29 @@ test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err ' +test_expect_success 'refuse to overwrite: worktree in rebase with --update-refs' ' + test_when_finished rm -rf .git/worktrees/wt-3/rebase-merge && + + mkdir -p .git/worktrees/wt-3/rebase-merge && + touch .git/worktrees/wt-3/rebase-merge/interactive && + + cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && + refs/heads/fake-3 + $(git rev-parse HEAD~1) + $(git rev-parse HEAD) + refs/heads/fake-4 + $(git rev-parse HEAD) + $(git rev-parse HEAD) + EOF + + for i in 3 4 + do + test_must_fail git branch -f fake-$i HEAD 2>err && + grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || + return 1 + done +' + test_expect_success !SANITIZE_LEAK 'refuse to fetch over ref: checked out' ' test_must_fail git fetch server +refs/heads/wt-3:refs/heads/wt-3 2>err && grep "refusing to fetch into branch '\''refs/heads/wt-3'\''" err && -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v5 04/12] rebase-interactive: update 'merge' description 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (2 preceding siblings ...) 2022-07-19 18:33 ` [PATCH v5 03/12] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 05/12] sequencer: define array with enum values Derrick Stolee via GitGitGadget ` (9 subsequent siblings) 13 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The 'merge' command description for the todo list documentation in an interactive rebase has multiple lines. The lines other than the first one start with dots ('.') while the similar multi-line documentation for 'fixup' does not. This description only appears in the comment text of the todo file during an interactive rebase. The 'merge' command was documented when interactive rebase was first ported to C in 145e05ac44b (rebase -i: rewrite append_todo_help() in C, 2018-08-10). These dots might have been carried over from the previous shell implementation. The 'fixup' command was documented more recently in 9e3cebd97cb (rebase -i: add fixup [-C | -c] command, 2021-01-29). Looking at the output in an editor, my personal opinion is that the dots are unnecessary and noisy. Remove them now before adding more commands with multi-line documentation. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- rebase-interactive.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rebase-interactive.c b/rebase-interactive.c index 87649d0c016..22394224faa 100644 --- a/rebase-interactive.c +++ b/rebase-interactive.c @@ -54,9 +54,9 @@ void append_todo_help(int command_count, "l, label <label> = label current HEAD with a name\n" "t, reset <label> = reset HEAD to a label\n" "m, merge [-C <commit> | -c <commit>] <label> [# <oneline>]\n" -". create a merge commit using the original merge commit's\n" -". message (or the oneline, if no original merge commit was\n" -". specified); use -c <commit> to reword the commit message\n" +" create a merge commit using the original merge commit's\n" +" message (or the oneline, if no original merge commit was\n" +" specified); use -c <commit> to reword the commit message\n" "\n" "These lines can be re-ordered; they are executed from top to bottom.\n"); unsigned edit_todo = !(shortrevisions && shortonto); -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v5 05/12] sequencer: define array with enum values 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (3 preceding siblings ...) 2022-07-19 18:33 ` [PATCH v5 04/12] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 06/12] sequencer: add update-ref command Derrick Stolee via GitGitGadget ` (8 subsequent siblings) 13 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The todo_command_info array defines which strings match with which todo_command enum values. The array is defined in the same order as the enum values, but if one changed without the other, then we would have unexpected results. Make it easier to see changes to the enum and this array by using the enum values as the indices of the array. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/sequencer.c b/sequencer.c index 7d131d06cc3..2711182e43f 100644 --- a/sequencer.c +++ b/sequencer.c @@ -1708,20 +1708,20 @@ static struct { char c; const char *str; } todo_command_info[] = { - { 'p', "pick" }, - { 0, "revert" }, - { 'e', "edit" }, - { 'r', "reword" }, - { 'f', "fixup" }, - { 's', "squash" }, - { 'x', "exec" }, - { 'b', "break" }, - { 'l', "label" }, - { 't', "reset" }, - { 'm', "merge" }, - { 0, "noop" }, - { 'd', "drop" }, - { 0, NULL } + [TODO_PICK] = { 'p', "pick" }, + [TODO_REVERT] = { 0, "revert" }, + [TODO_EDIT] = { 'e', "edit" }, + [TODO_REWORD] = { 'r', "reword" }, + [TODO_FIXUP] = { 'f', "fixup" }, + [TODO_SQUASH] = { 's', "squash" }, + [TODO_EXEC] = { 'x', "exec" }, + [TODO_BREAK] = { 'b', "break" }, + [TODO_LABEL] = { 'l', "label" }, + [TODO_RESET] = { 't', "reset" }, + [TODO_MERGE] = { 'm', "merge" }, + [TODO_NOOP] = { 0, "noop" }, + [TODO_DROP] = { 'd', "drop" }, + [TODO_COMMENT] = { 0, NULL }, }; static const char *command_to_string(const enum todo_command command) -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v5 06/12] sequencer: add update-ref command 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (4 preceding siblings ...) 2022-07-19 18:33 ` [PATCH v5 05/12] sequencer: define array with enum values Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 07/12] rebase: add --update-refs option Derrick Stolee via GitGitGadget ` (7 subsequent siblings) 13 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> Add the boilerplate for an "update-ref" command in the sequencer. This connects to the current no-op do_update_ref() which will be filled in after more connections are created. The syntax in the todo list will be "update-ref <ref-name>" to signal that we should store the current commit as the value for updating <ref-name> at the end of the rebase. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- rebase-interactive.c | 3 +++ sequencer.c | 14 +++++++++++++- sequencer.h | 1 + 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/rebase-interactive.c b/rebase-interactive.c index 22394224faa..1ff07647af3 100644 --- a/rebase-interactive.c +++ b/rebase-interactive.c @@ -57,6 +57,9 @@ void append_todo_help(int command_count, " create a merge commit using the original merge commit's\n" " message (or the oneline, if no original merge commit was\n" " specified); use -c <commit> to reword the commit message\n" +"u, update-ref <ref> = track a placeholder for the <ref> to be updated\n" +" to this position in the new commits. The <ref> is\n" +" updated at the end of the rebase\n" "\n" "These lines can be re-ordered; they are executed from top to bottom.\n"); unsigned edit_todo = !(shortrevisions && shortonto); diff --git a/sequencer.c b/sequencer.c index 2711182e43f..0dc9c05c5bb 100644 --- a/sequencer.c +++ b/sequencer.c @@ -1719,6 +1719,7 @@ static struct { [TODO_LABEL] = { 'l', "label" }, [TODO_RESET] = { 't', "reset" }, [TODO_MERGE] = { 'm', "merge" }, + [TODO_UPDATE_REF] = { 'u', "update-ref" }, [TODO_NOOP] = { 0, "noop" }, [TODO_DROP] = { 'd', "drop" }, [TODO_COMMENT] = { 0, NULL }, @@ -2480,7 +2481,7 @@ static int parse_insn_line(struct repository *r, struct todo_item *item, command_to_string(item->command)); if (item->command == TODO_EXEC || item->command == TODO_LABEL || - item->command == TODO_RESET) { + item->command == TODO_RESET || item->command == TODO_UPDATE_REF) { item->commit = NULL; item->arg_offset = bol - buf; item->arg_len = (int)(eol - bol); @@ -4079,6 +4080,11 @@ leave_merge: return ret; } +static int do_update_ref(struct repository *r, const char *ref_name) +{ + return 0; +} + static int is_final_fixup(struct todo_list *todo_list) { int i = todo_list->current; @@ -4454,6 +4460,12 @@ static int pick_commits(struct repository *r, return error_with_patch(r, item->commit, arg, item->arg_len, opts, res, 0); + } else if (item->command == TODO_UPDATE_REF) { + struct strbuf ref = STRBUF_INIT; + strbuf_add(&ref, arg, item->arg_len); + if ((res = do_update_ref(r, ref.buf))) + reschedule = 1; + strbuf_release(&ref); } else if (!is_noop(item->command)) return error(_("unknown command %d"), item->command); diff --git a/sequencer.h b/sequencer.h index 3ae541bb145..2cf5c1b9a38 100644 --- a/sequencer.h +++ b/sequencer.h @@ -95,6 +95,7 @@ enum todo_command { TODO_LABEL, TODO_RESET, TODO_MERGE, + TODO_UPDATE_REF, /* commands that do nothing but are counted for reporting progress */ TODO_NOOP, TODO_DROP, -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v5 07/12] rebase: add --update-refs option 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (5 preceding siblings ...) 2022-07-19 18:33 ` [PATCH v5 06/12] sequencer: add update-ref command Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 08/12] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget ` (6 subsequent siblings) 13 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> When working on a large feature, it can be helpful to break that feature into multiple smaller parts that become reviewed in sequence. During development or during review, a change to one part of the feature could affect multiple of these parts. An interactive rebase can help adjust the multi-part "story" of the branch. However, if there are branches tracking the different parts of the feature, then rebasing the entire list of commits can create commits not reachable from those "sub branches". It can take a manual step to update those branches. Add a new --update-refs option to 'git rebase -i' that adds 'update-ref <ref>' steps to the todo file whenever a commit that is being rebased is decorated with that <ref>. At the very end, the rebase process updates all of the listed refs to the values stored during the rebase operation. Be sure to iterate after any squashing or fixups are placed. Update the branch only after those squashes and fixups are complete. This allows a --fixup commit at the tip of the feature to apply correctly to the sub branch, even if it is fixing up the most-recent commit in that part. This change update the documentation and builtin to accept the --update-refs option as well as updating the todo file with the 'update-ref' commands. Tests are added to ensure that these todo commands are added in the correct locations. This change does _not_ include the actual behavior of tracking the updated refs and writing the new ref values at the end of the rebase process. That is deferred to a later change. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- Documentation/git-rebase.txt | 7 +++ builtin/rebase.c | 5 ++ sequencer.c | 107 ++++++++++++++++++++++++++++++++++ sequencer.h | 1 + t/t2407-worktree-heads.sh | 22 +++++++ t/t3404-rebase-interactive.sh | 74 +++++++++++++++++++++++ 6 files changed, 216 insertions(+) diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index 262fb01aec0..dfd847d6615 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -609,6 +609,12 @@ provided. Otherwise an explicit `--no-reschedule-failed-exec` at the start would be overridden by the presence of `rebase.rescheduleFailedExec=true` configuration. +--update-refs:: +--no-update-refs:: + Automatically force-update any branches that point to commits that + are being rebased. Any branches that are checked out in a worktree + are not updated in this way. + INCOMPATIBLE OPTIONS -------------------- @@ -632,6 +638,7 @@ are incompatible with the following options: * --empty= * --reapply-cherry-picks * --edit-todo + * --update-refs * --root when used in combination with --onto In addition, the following pairs of options are incompatible: diff --git a/builtin/rebase.c b/builtin/rebase.c index 7ab50cda2ad..fd64897132a 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -102,6 +102,7 @@ struct rebase_options { int reschedule_failed_exec; int reapply_cherry_picks; int fork_point; + int update_refs; }; #define REBASE_OPTIONS_INIT { \ @@ -298,6 +299,7 @@ static int do_interactive_rebase(struct rebase_options *opts, unsigned flags) ret = complete_action(the_repository, &replay, flags, shortrevisions, opts->onto_name, opts->onto, &opts->orig_head, &commands, opts->autosquash, + opts->update_refs, &todo_list); } @@ -1124,6 +1126,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) OPT_BOOL(0, "autosquash", &options.autosquash, N_("move commits that begin with " "squash!/fixup! under -i")), + OPT_BOOL(0, "update-refs", &options.update_refs, + N_("update branches that point to commits " + "that are being rebased")), { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"), N_("GPG-sign commits"), PARSE_OPT_OPTARG, NULL, (intptr_t) "" }, diff --git a/sequencer.c b/sequencer.c index 0dc9c05c5bb..a4ee2799d6f 100644 --- a/sequencer.c +++ b/sequencer.c @@ -35,6 +35,7 @@ #include "commit-reach.h" #include "rebase-interactive.h" #include "reset.h" +#include "branch.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -5638,10 +5639,113 @@ static int skip_unnecessary_picks(struct repository *r, return 0; } +struct todo_add_branch_context { + struct todo_item *items; + size_t items_nr; + size_t items_alloc; + struct strbuf *buf; + struct commit *commit; + struct string_list refs_to_oids; +}; + +static int add_decorations_to_list(const struct commit *commit, + struct todo_add_branch_context *ctx) +{ + const struct name_decoration *decoration = get_name_decoration(&commit->object); + + while (decoration) { + struct todo_item *item; + const char *path; + size_t base_offset = ctx->buf->len; + + ALLOC_GROW(ctx->items, + ctx->items_nr + 1, + ctx->items_alloc); + item = &ctx->items[ctx->items_nr]; + memset(item, 0, sizeof(*item)); + + /* If the branch is checked out, then leave a comment instead. */ + if ((path = branch_checked_out(decoration->name))) { + item->command = TODO_COMMENT; + strbuf_addf(ctx->buf, "# Ref %s checked out at '%s'\n", + decoration->name, path); + } else { + struct string_list_item *sti; + item->command = TODO_UPDATE_REF; + strbuf_addf(ctx->buf, "%s\n", decoration->name); + + sti = string_list_insert(&ctx->refs_to_oids, + decoration->name); + sti->util = oiddup(the_hash_algo->null_oid); + } + + item->offset_in_buf = base_offset; + item->arg_offset = base_offset; + item->arg_len = ctx->buf->len - base_offset; + ctx->items_nr++; + + decoration = decoration->next; + } + + return 0; +} + +/* + * For each 'pick' command, find out if the commit has a decoration in + * refs/heads/. If so, then add a 'label for-update-refs/' command. + */ +static int todo_list_add_update_ref_commands(struct todo_list *todo_list) +{ + int i; + static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP; + static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP; + struct decoration_filter decoration_filter = { + .include_ref_pattern = &decorate_refs_include, + .exclude_ref_pattern = &decorate_refs_exclude, + .exclude_ref_config_pattern = &decorate_refs_exclude_config, + }; + struct todo_add_branch_context ctx = { + .buf = &todo_list->buf, + .refs_to_oids = STRING_LIST_INIT_DUP, + }; + + ctx.items_alloc = 2 * todo_list->nr + 1; + ALLOC_ARRAY(ctx.items, ctx.items_alloc); + + string_list_append(&decorate_refs_include, "refs/heads/"); + load_ref_decorations(&decoration_filter, 0); + + for (i = 0; i < todo_list->nr; ) { + struct todo_item *item = &todo_list->items[i]; + + /* insert ith item into new list */ + ALLOC_GROW(ctx.items, + ctx.items_nr + 1, + ctx.items_alloc); + + ctx.items[ctx.items_nr++] = todo_list->items[i++]; + + if (item->commit) { + ctx.commit = item->commit; + add_decorations_to_list(item->commit, &ctx); + } + } + + string_list_clear(&ctx.refs_to_oids, 1); + free(todo_list->items); + todo_list->items = ctx.items; + todo_list->nr = ctx.items_nr; + todo_list->alloc = ctx.items_alloc; + + return 0; +} + int complete_action(struct repository *r, struct replay_opts *opts, unsigned flags, const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, + unsigned update_refs, struct todo_list *todo_list) { char shortonto[GIT_MAX_HEXSZ + 1]; @@ -5660,6 +5764,9 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla item->arg_len = item->arg_offset = item->flags = item->offset_in_buf = 0; } + if (update_refs && todo_list_add_update_ref_commands(todo_list)) + return -1; + if (autosquash && todo_list_rearrange_squash(todo_list)) return -1; diff --git a/sequencer.h b/sequencer.h index 2cf5c1b9a38..e671d7e0743 100644 --- a/sequencer.h +++ b/sequencer.h @@ -167,6 +167,7 @@ int complete_action(struct repository *r, struct replay_opts *opts, unsigned fla const char *shortrevisions, const char *onto_name, struct commit *onto, const struct object_id *orig_head, struct string_list *commands, unsigned autosquash, + unsigned update_refs, struct todo_list *todo_list); int todo_list_rearrange_squash(struct todo_list *todo_list); diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index 97f5c87f8c8..8a03f14df8d 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -164,4 +164,26 @@ test_expect_success 'refuse to overwrite when in error states' ' done ' +. "$TEST_DIRECTORY"/lib-rebase.sh + +test_expect_success !SANITIZE_LEAK 'refuse to overwrite during rebase with --update-refs' ' + git commit --fixup HEAD~2 --allow-empty && + ( + set_cat_todo_editor && + test_must_fail git rebase -i --update-refs HEAD~3 >todo && + ! grep "update-refs" todo + ) && + git branch -f allow-update HEAD~2 && + ( + set_cat_todo_editor && + test_must_fail git rebase -i --update-refs HEAD~3 >todo && + grep "update-ref refs/heads/allow-update" todo + ) +' + +# This must be the last test in this file +test_expect_success '$EDITOR and friends are unchanged' ' + test_editor_unchanged +' + test_done diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index f31afd4a547..75c0b90fae1 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1743,6 +1743,80 @@ test_expect_success 'ORIG_HEAD is updated correctly' ' test_cmp_rev ORIG_HEAD test-orig-head@{1} ' +test_expect_success '--update-refs adds label and update-ref commands' ' + git checkout -b update-refs no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git commit --allow-empty --fixup=third && + git branch -f is-not-reordered && + git commit --allow-empty --fixup=HEAD~4 && + git branch -f shared-tip && + ( + set_cat_todo_editor && + + cat >expect <<-EOF && + pick $(git log -1 --format=%h J) J + fixup $(git log -1 --format=%h update-refs) fixup! J # empty + update-ref refs/heads/second + update-ref refs/heads/first + pick $(git log -1 --format=%h K) K + pick $(git log -1 --format=%h L) L + fixup $(git log -1 --format=%h is-not-reordered) fixup! L # empty + update-ref refs/heads/third + pick $(git log -1 --format=%h M) M + update-ref refs/heads/no-conflict-branch + update-ref refs/heads/is-not-reordered + update-ref refs/heads/shared-tip + EOF + + test_must_fail git rebase -i --autosquash --update-refs primary >todo && + test_cmp expect todo + ) +' + +test_expect_success '--update-refs adds commands with --rebase-merges' ' + git checkout -b update-refs-with-merge no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git merge -m merge branch2 && + git branch -f merge-branch && + git commit --fixup=third --allow-empty && + ( + set_cat_todo_editor && + + cat >expect <<-EOF && + label onto + reset onto + pick $(git log -1 --format=%h branch2~1) F + pick $(git log -1 --format=%h branch2) I + update-ref refs/heads/branch2 + label merge + reset onto + pick $(git log -1 --format=%h refs/heads/second) J + update-ref refs/heads/second + update-ref refs/heads/first + pick $(git log -1 --format=%h refs/heads/third~1) K + pick $(git log -1 --format=%h refs/heads/third) L + fixup $(git log -1 --format=%h update-refs-with-merge) fixup! L # empty + update-ref refs/heads/third + pick $(git log -1 --format=%h HEAD~2) M + update-ref refs/heads/no-conflict-branch + merge -C $(git log -1 --format=%h HEAD~1) merge # merge + update-ref refs/heads/merge-branch + EOF + + test_must_fail git rebase -i --autosquash \ + --rebase-merges=rebase-cousins \ + --update-refs primary >todo && + + test_cmp expect todo + ) +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v5 08/12] rebase: update refs from 'update-ref' commands 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (6 preceding siblings ...) 2022-07-19 18:33 ` [PATCH v5 07/12] rebase: add --update-refs option Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-21 14:03 ` Phillip Wood 2022-07-19 18:33 ` [PATCH v5 09/12] sequencer: rewrite update-refs as user edits todo list Derrick Stolee via GitGitGadget ` (5 subsequent siblings) 13 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The previous change introduced the 'git rebase --update-refs' option which added 'update-ref <ref>' commands to the todo list of an interactive rebase. Teach Git to record the HEAD position when reaching these 'update-ref' commands. The ref/before/after triple is stored in the $GIT_DIR/rebase-merge/update-refs file. A previous change parsed this file to avoid having other processes updating the refs in that file while the rebase is in progress. Not only do we update the file when the sequencer reaches these 'update-ref' commands, we then update the refs themselves at the end of the rebase sequence. If the rebase is aborted before this final step, then the refs are not updated. The 'before' value is used to ensure that we do not accidentally obliterate a ref that was updated concurrently (say, by an older version of Git or a third-party tool). Now that the 'git rebase --update-refs' command is implemented to write to the update-refs file, we can remove the fake construction of the update-refs file from a test in t2407-worktree-heads.sh. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 120 +++++++++++++++++++++++++++++++++- t/t2407-worktree-heads.sh | 21 ++---- t/t3404-rebase-interactive.sh | 17 +++++ 3 files changed, 140 insertions(+), 18 deletions(-) diff --git a/sequencer.c b/sequencer.c index a4ee2799d6f..ce8c7d8cd3a 100644 --- a/sequencer.c +++ b/sequencer.c @@ -36,6 +36,7 @@ #include "rebase-interactive.h" #include "reset.h" #include "branch.h" +#include "log-tree.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -193,6 +194,21 @@ struct update_ref_record { struct object_id after; }; +static struct update_ref_record *init_update_ref_record(const char *ref) +{ + struct update_ref_record *rec; + + CALLOC_ARRAY(rec, 1); + + oidcpy(&rec->before, null_oid()); + oidcpy(&rec->after, null_oid()); + + /* This may fail, but that's fine, we will keep the null OID. */ + read_ref(ref, &rec->before); + + return rec; +} + static int git_sequencer_config(const char *k, const char *v, void *cb) { struct replay_opts *opts = cb; @@ -4081,11 +4097,97 @@ leave_merge: return ret; } -static int do_update_ref(struct repository *r, const char *ref_name) +static int write_update_refs_state(struct string_list *refs_to_oids) +{ + int result = 0; + struct lock_file lock = LOCK_INIT; + FILE *fp = NULL; + struct string_list_item *item; + char *path; + + if (!refs_to_oids->nr) + return 0; + + path = rebase_path_update_refs(the_repository->gitdir); + + if (safe_create_leading_directories(path)) { + result = error(_("unable to create leading directories of %s"), + path); + goto cleanup; + } + + if (hold_lock_file_for_update(&lock, path, 0) < 0) { + result = error(_("another 'rebase' process appears to be running; " + "'%s.lock' already exists"), + path); + goto cleanup; + } + + fp = fdopen_lock_file(&lock, "w"); + if (!fp) { + result = error_errno(_("could not open '%s' for writing"), path); + rollback_lock_file(&lock); + goto cleanup; + } + + for_each_string_list_item(item, refs_to_oids) { + struct update_ref_record *rec = item->util; + fprintf(fp, "%s\n%s\n%s\n", item->string, + oid_to_hex(&rec->before), oid_to_hex(&rec->after)); + } + + result = commit_lock_file(&lock); + +cleanup: + free(path); + return result; +} + +static int do_update_ref(struct repository *r, const char *refname) { + struct string_list_item *item; + struct string_list list = STRING_LIST_INIT_DUP; + + if (sequencer_get_update_refs_state(r->gitdir, &list)) + return -1; + + for_each_string_list_item(item, &list) { + if (!strcmp(item->string, refname)) { + struct update_ref_record *rec = item->util; + if (read_ref("HEAD", &rec->after)) + return -1; + break; + } + } + + write_update_refs_state(&list); + string_list_clear(&list, 1); return 0; } +static int do_update_refs(struct repository *r) +{ + int res = 0; + struct string_list_item *item; + struct string_list refs_to_oids = STRING_LIST_INIT_DUP; + struct ref_store *refs = get_main_ref_store(r); + + if ((res = sequencer_get_update_refs_state(r->gitdir, &refs_to_oids))) + return res; + + for_each_string_list_item(item, &refs_to_oids) { + struct update_ref_record *rec = item->util; + + res |= refs_update_ref(refs, "rewritten during rebase", + item->string, + &rec->after, &rec->before, + 0, UPDATE_REFS_MSG_ON_ERR); + } + + string_list_clear(&refs_to_oids, 1); + return res; +} + static int is_final_fixup(struct todo_list *todo_list) { int i = todo_list->current; @@ -4601,6 +4703,9 @@ cleanup_head_ref: strbuf_release(&buf); strbuf_release(&head_ref); + + if (do_update_refs(r)) + return -1; } /* @@ -5676,7 +5781,7 @@ static int add_decorations_to_list(const struct commit *commit, sti = string_list_insert(&ctx->refs_to_oids, decoration->name); - sti->util = oiddup(the_hash_algo->null_oid); + sti->util = init_update_ref_record(decoration->name); } item->offset_in_buf = base_offset; @@ -5696,7 +5801,7 @@ static int add_decorations_to_list(const struct commit *commit, */ static int todo_list_add_update_ref_commands(struct todo_list *todo_list) { - int i; + int i, res; static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP; static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP; @@ -5732,7 +5837,16 @@ static int todo_list_add_update_ref_commands(struct todo_list *todo_list) } } + res = write_update_refs_state(&ctx.refs_to_oids); + string_list_clear(&ctx.refs_to_oids, 1); + + if (res) { + /* we failed, so clean up the new list. */ + free(ctx.items); + return res; + } + free(todo_list->items); todo_list->items = ctx.items; todo_list->nr = ctx.items_nr; diff --git a/t/t2407-worktree-heads.sh b/t/t2407-worktree-heads.sh index 8a03f14df8d..50815acd3e8 100755 --- a/t/t2407-worktree-heads.sh +++ b/t/t2407-worktree-heads.sh @@ -81,25 +81,16 @@ test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer grep "cannot force update the branch '\''wt-2'\'' checked out at.*wt-2" err ' -test_expect_success 'refuse to overwrite: worktree in rebase with --update-refs' ' - test_when_finished rm -rf .git/worktrees/wt-3/rebase-merge && - - mkdir -p .git/worktrees/wt-3/rebase-merge && - touch .git/worktrees/wt-3/rebase-merge/interactive && +test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase with --update-refs' ' + test_when_finished git -C wt-3 rebase --abort && - cat >.git/worktrees/wt-3/rebase-merge/update-refs <<-EOF && - refs/heads/fake-3 - $(git rev-parse HEAD~1) - $(git rev-parse HEAD) - refs/heads/fake-4 - $(git rev-parse HEAD) - $(git rev-parse HEAD) - EOF + git branch -f can-be-updated wt-3 && + test_must_fail git -C wt-3 rebase --update-refs conflict-3 && for i in 3 4 do - test_must_fail git branch -f fake-$i HEAD 2>err && - grep "cannot force update the branch '\''fake-$i'\'' checked out at.*wt-3" err || + test_must_fail git branch -f can-be-updated HEAD 2>err && + grep "cannot force update the branch '\''can-be-updated'\'' checked out at.*wt-3" err || return 1 done ' diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 75c0b90fae1..2fffcf5d5fb 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1817,6 +1817,23 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' ) ' +test_expect_success '--update-refs updates refs correctly' ' + git checkout -B update-refs no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + test_commit extra2 fileX && + git commit --amend --fixup=L && + + git rebase -i --autosquash --update-refs primary && + + test_cmp_rev HEAD~3 refs/heads/first && + test_cmp_rev HEAD~3 refs/heads/second && + test_cmp_rev HEAD~1 refs/heads/third && + test_cmp_rev HEAD refs/heads/no-conflict-branch +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v5 08/12] rebase: update refs from 'update-ref' commands 2022-07-19 18:33 ` [PATCH v5 08/12] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget @ 2022-07-21 14:03 ` Phillip Wood 0 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-07-21 14:03 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, SZEDER Gábor, Derrick Stolee Hi Stolee On 19/07/2022 19:33, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > +static int do_update_ref(struct repository *r, const char *refname) > { > + struct string_list_item *item; > + struct string_list list = STRING_LIST_INIT_DUP; > + > + if (sequencer_get_update_refs_state(r->gitdir, &list)) > + return -1; This is good, we now return an error if we cannot read the update-refs file. > + for_each_string_list_item(item, &list) { > + if (!strcmp(item->string, refname)) { > + struct update_ref_record *rec = item->util; > + if (read_ref("HEAD", &rec->after)) > + return -1; > + break; > + } > + } > + > + write_update_refs_state(&list); The return value of write_update_refs_state() should be propagated to the caller. > + string_list_clear(&list, 1); > return 0; > } Best Wishes Phillip ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v5 09/12] sequencer: rewrite update-refs as user edits todo list 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (7 preceding siblings ...) 2022-07-19 18:33 ` [PATCH v5 08/12] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-21 14:04 ` Phillip Wood 2022-07-19 18:33 ` [PATCH v5 10/12] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget ` (4 subsequent siblings) 13 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> An interactive rebase provides opportunities for the user to edit the todo list. The --update-refs option initializes the list with some 'update-ref <ref>' steps, but the user could add these manually. Further, the user could add or remove these steps during pauses in the interactive rebase. Add a new method, todo_list_filter_update_refs(), that scans a todo_list and compares it to the stored update-refs file. There are two actions that can happen at this point: 1. If a '<ref>/<before>/<after>' triple in the update-refs file does not have a matching 'update-ref <ref>' command in the todo-list _and_ the <after> value is the null OID, then remove that triple. Here, the user removed the 'update-ref <ref>' command before it was executed, since if it was executed then the <after> value would store the commit at that position. 2. If a 'update-ref <ref>' command in the todo-list does not have a matching '<ref>/<before>/<after>' triple in the update-refs file, then insert a new one. Store the <before> value to be the current OID pointed at by <ref>. This is handled inside of the init_update_ref_record() helper method. We can test that this works by rewriting the todo-list several times in the course of a rebase. Check that each ref is locked or unlocked for updates after each todo-list update. We can also verify that the ref update fails if a concurrent process updates one of the refs after the rebase process records the "locked" ref location. To help these tests, add a new 'set_replace_editor' helper that will replace the todo-list with an exact file. Reported-by: Phillip Wood <phillip.wood123@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- rebase-interactive.c | 6 ++ sequencer.c | 96 +++++++++++++++++++++++ sequencer.h | 12 +++ t/lib-rebase.sh | 15 ++++ t/t3404-rebase-interactive.sh | 139 ++++++++++++++++++++++++++++++++++ 5 files changed, 268 insertions(+) diff --git a/rebase-interactive.c b/rebase-interactive.c index 1ff07647af3..7407c593191 100644 --- a/rebase-interactive.c +++ b/rebase-interactive.c @@ -146,6 +146,12 @@ int edit_todo_list(struct repository *r, struct todo_list *todo_list, return -4; } + /* + * See if branches need to be added or removed from the update-refs + * file based on the new todo list. + */ + todo_list_filter_update_refs(r, new_todo); + return 0; } diff --git a/sequencer.c b/sequencer.c index ce8c7d8cd3a..67812c0294f 100644 --- a/sequencer.c +++ b/sequencer.c @@ -4143,6 +4143,102 @@ cleanup: return result; } +/* + * Parse the update-refs file for the current rebase, then remove the + * refs that do not appear in the todo_list (and have not had updated + * values stored) and add refs that are in the todo_list but not + * represented in the update-refs file. + * + * If there are changes to the update-refs list, then write the new state + * to disk. + */ +void todo_list_filter_update_refs(struct repository *r, + struct todo_list *todo_list) +{ + int i; + int updated = 0; + struct string_list update_refs = STRING_LIST_INIT_DUP; + + sequencer_get_update_refs_state(r->gitdir, &update_refs); + + /* + * For each item in the update_refs list, if it has no updated + * value and does not appear in the todo_list, then remove it + * from the update_refs list. + */ + for (i = 0; i < update_refs.nr; i++) { + int j; + int found = 0; + const char *ref = update_refs.items[i].string; + size_t reflen = strlen(ref); + struct update_ref_record *rec = update_refs.items[i].util; + + /* OID already stored as updated. */ + if (!is_null_oid(&rec->after)) + continue; + + for (j = 0; !found && j < todo_list->total_nr; j++) { + struct todo_item *item = &todo_list->items[j]; + const char *arg = todo_list->buf.buf + item->arg_offset; + + if (item->command != TODO_UPDATE_REF) + continue; + + if (item->arg_len != reflen || + strncmp(arg, ref, reflen)) + continue; + + found = 1; + } + + if (!found) { + free(update_refs.items[i].string); + free(update_refs.items[i].util); + + update_refs.nr--; + MOVE_ARRAY(update_refs.items + i, update_refs.items + i + 1, update_refs.nr - i); + + updated = 1; + i--; + } + } + + /* + * For each todo_item, check if its ref is in the update_refs list. + * If not, then add it as an un-updated ref. + */ + for (i = 0; i < todo_list->total_nr; i++) { + struct todo_item *item = &todo_list->items[i]; + const char *arg = todo_list->buf.buf + item->arg_offset; + int j, found = 0; + + if (item->command != TODO_UPDATE_REF) + continue; + + for (j = 0; !found && j < update_refs.nr; j++) { + const char *ref = update_refs.items[j].string; + + found = strlen(ref) == item->arg_len && + !strncmp(ref, arg, item->arg_len); + } + + if (!found) { + struct string_list_item *inserted; + struct strbuf argref = STRBUF_INIT; + + strbuf_add(&argref, arg, item->arg_len); + inserted = string_list_insert(&update_refs, argref.buf); + inserted->util = init_update_ref_record(argref.buf); + strbuf_release(&argref); + updated = 1; + } + } + + if (updated) + write_update_refs_state(&update_refs); + string_list_clear(&update_refs, 1); +} + static int do_update_ref(struct repository *r, const char *refname) { struct string_list_item *item; diff --git a/sequencer.h b/sequencer.h index e671d7e0743..98f3bcc1658 100644 --- a/sequencer.h +++ b/sequencer.h @@ -132,6 +132,18 @@ void todo_list_release(struct todo_list *todo_list); const char *todo_item_get_arg(struct todo_list *todo_list, struct todo_item *item); +/* + * Parse the update-refs file for the current rebase, then remove the + * refs that do not appear in the todo_list (and have not had updated + * values stored) and add refs that are in the todo_list but not + * represented in the update-refs file. + * + * If there are changes to the update-refs list, then write the new state + * to disk. + */ +void todo_list_filter_update_refs(struct repository *r, + struct todo_list *todo_list); + /* Call this to setup defaults before parsing command line options */ void sequencer_init_config(struct replay_opts *opts); int sequencer_pick_revisions(struct repository *repo, diff --git a/t/lib-rebase.sh b/t/lib-rebase.sh index ec6b9b107da..b57541356bd 100644 --- a/t/lib-rebase.sh +++ b/t/lib-rebase.sh @@ -207,3 +207,18 @@ check_reworded_commits () { >reword-log && test_cmp reword-expected reword-log } + +# usage: set_replace_editor <file> +# +# Replace the todo file with the exact contents of the given file. +set_replace_editor () { + cat >script <<-\EOF && + cat FILENAME >"$1" + + echo 'rebase -i script after editing:' + cat "$1" + EOF + + sed -e "s/FILENAME/$1/g" <script | write_script fake-editor.sh && + test_set_editor "$(pwd)/fake-editor.sh" +} diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 2fffcf5d5fb..bebf9b4def7 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1834,6 +1834,145 @@ test_expect_success '--update-refs updates refs correctly' ' test_cmp_rev HEAD refs/heads/no-conflict-branch ' +test_expect_success 'respect user edits to update-ref steps' ' + git checkout -B update-refs-break no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~3 && + git branch -f third HEAD~1 && + git branch -f unseen base && + + # First, we will add breaks to the expected todo file + cat >fake-todo-1 <<-EOF && + pick $(git rev-parse HEAD~3) + break + update-ref refs/heads/second + update-ref refs/heads/first + + pick $(git rev-parse HEAD~2) + pick $(git rev-parse HEAD~1) + update-ref refs/heads/third + + pick $(git rev-parse HEAD) + update-ref refs/heads/no-conflict-branch + EOF + + # Second, we will drop some update-refs commands (and move one) + cat >fake-todo-2 <<-EOF && + update-ref refs/heads/second + + pick $(git rev-parse HEAD~2) + update-ref refs/heads/third + pick $(git rev-parse HEAD~1) + break + + pick $(git rev-parse HEAD) + EOF + + # Third, we will: + # * insert a new one (new-branch), + # * re-add an old one (first), and + # * add a second instance of a previously-stored one (second) + cat >fake-todo-3 <<-EOF && + update-ref refs/heads/unseen + update-ref refs/heads/new-branch + pick $(git rev-parse HEAD) + update-ref refs/heads/first + update-ref refs/heads/second + EOF + + ( + set_replace_editor fake-todo-1 && + git rebase -i --update-refs primary && + + # These branches are currently locked. + for b in first second third no-conflict-branch + do + test_must_fail git branch -f $b base || return 1 + done && + + set_replace_editor fake-todo-2 && + git rebase --edit-todo && + + # These branches are currently locked. + for b in second third + do + test_must_fail git branch -f $b base || return 1 + done && + + # These branches are currently unlocked for checkout. + for b in first no-conflict-branch + do + git worktree add wt-$b $b && + git worktree remove wt-$b || return 1 + done && + + git rebase --continue && + + set_replace_editor fake-todo-3 && + git rebase --edit-todo && + + # These branches are currently locked. + for b in second third first unseen + do + test_must_fail git branch -f $b base || return 1 + done && + + # These branches are currently unlocked for checkout. + for b in no-conflict-branch + do + git worktree add wt-$b $b && + git worktree remove wt-$b || return 1 + done && + + git rebase --continue + ) && + + test_cmp_rev HEAD~2 refs/heads/third && + test_cmp_rev HEAD~1 refs/heads/unseen && + test_cmp_rev HEAD~1 refs/heads/new-branch && + test_cmp_rev HEAD refs/heads/first && + test_cmp_rev HEAD refs/heads/second && + test_cmp_rev HEAD refs/heads/no-conflict-branch +' + +test_expect_success '--update-refs: check failed ref update' ' + git checkout -B update-refs-error no-conflict-branch && + git branch -f base HEAD~4 && + git branch -f first HEAD~3 && + git branch -f second HEAD~2 && + git branch -f third HEAD~1 && + + cat >fake-todo <<-EOF && + pick $(git rev-parse HEAD~3) + break + update-ref refs/heads/first + + pick $(git rev-parse HEAD~2) + update-ref refs/heads/second + + pick $(git rev-parse HEAD~1) + update-ref refs/heads/third + + pick $(git rev-parse HEAD) + update-ref refs/heads/no-conflict-branch + EOF + + ( + set_replace_editor fake-todo && + git rebase -i --update-refs base + ) && + + # At this point, the values of first, second, and third are + # recorded in the update-refs file. We will force-update the + # "second" ref, but "git branch -f" will not work because of + # the lock in the update-refs file. + git rev-parse third >.git/refs/heads/second && + + git rebase --continue 2>err && + grep "update_ref failed for ref '\''refs/heads/second'\''" err +' + # This must be the last test in this file test_expect_success '$EDITOR and friends are unchanged' ' test_editor_unchanged -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v5 09/12] sequencer: rewrite update-refs as user edits todo list 2022-07-19 18:33 ` [PATCH v5 09/12] sequencer: rewrite update-refs as user edits todo list Derrick Stolee via GitGitGadget @ 2022-07-21 14:04 ` Phillip Wood 0 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-07-21 14:04 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, SZEDER Gábor, Derrick Stolee Hi Stolee On 19/07/2022 19:33, Derrick Stolee via GitGitGadget wrote: > From: Derrick Stolee <derrickstolee@github.com> > > +void todo_list_filter_update_refs(struct repository *r, > + struct todo_list *todo_list) > +{ > + int i; > + int updated = 0; > + struct string_list update_refs = STRING_LIST_INIT_DUP; > + > + sequencer_get_update_refs_state(r->gitdir, &update_refs); > + > + /* > + * For each item in the update_refs list, if it has no updated > + * value and does not appear in the todo_list, then remove it > + * from the update_refs list. > + */ > + for (i = 0; i < update_refs.nr; i++) { > + int j; > + int found = 0; > + const char *ref = update_refs.items[i].string; > + size_t reflen = strlen(ref); > + struct update_ref_record *rec = update_refs.items[i].util; > + > + /* OID already stored as updated. */ > + if (!is_null_oid(&rec->after)) > + continue; > + > + for (j = 0; !found && j < todo_list->total_nr; j++) { > + struct todo_item *item = &todo_list->items[j]; > + const char *arg = todo_list->buf.buf + item->arg_offset; > + > + if (item->command != TODO_UPDATE_REF) > + continue; > + > + if (item->arg_len != reflen || > + strncmp(arg, ref, reflen)) > + continue; > + > + found = 1; > + } > + > + if (!found) { > + free(update_refs.items[i].string); > + free(update_refs.items[i].util); > + > + update_refs.nr--; > + MOVE_ARRAY(update_refs.items + i, update_refs.items + i + 1, update_refs.nr - i); > + > + updated = 1; > + i--; > + } > + } > + > + /* > + * For each todo_item, check if its ref is in the update_refs list. > + * If not, then add it as an un-updated ref. > + */ > + for (i = 0; i < todo_list->total_nr; i++) { > + struct todo_item *item = &todo_list->items[i]; > + const char *arg = todo_list->buf.buf + item->arg_offset; > + int j, found = 0; > + > + if (item->command != TODO_UPDATE_REF) > + continue; > + > + for (j = 0; !found && j < update_refs.nr; j++) { > + const char *ref = update_refs.items[j].string; > + > + found = strlen(ref) == item->arg_len && > + !strncmp(ref, arg, item->arg_len); > + } > + > + if (!found) { > + struct string_list_item *inserted; > + struct strbuf argref = STRBUF_INIT; > + > + strbuf_add(&argref, arg, item->arg_len); > + inserted = string_list_insert(&update_refs, argref.buf); > + inserted->util = init_update_ref_record(argref.buf); > + strbuf_release(&argref); > + updated = 1; > + } > + } > + > + if (updated) > + write_update_refs_state(&update_refs); We should return an error if write_update_refs_state() fails Best Wishes Phillip ^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH v5 10/12] rebase: add rebase.updateRefs config option 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (8 preceding siblings ...) 2022-07-19 18:33 ` [PATCH v5 09/12] sequencer: rewrite update-refs as user edits todo list Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 11/12] sequencer: ignore HEAD ref under --update-refs Derrick Stolee via GitGitGadget ` (3 subsequent siblings) 13 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> The previous change added the --update-refs command-line option. For users who always want this mode, create the rebase.updateRefs config option which behaves the same way as rebase.autoSquash does with the --autosquash option. Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- Documentation/config/rebase.txt | 3 +++ Documentation/git-rebase.txt | 3 +++ builtin/rebase.c | 5 +++++ t/t3404-rebase-interactive.sh | 14 ++++++++++++++ 4 files changed, 25 insertions(+) diff --git a/Documentation/config/rebase.txt b/Documentation/config/rebase.txt index 8c979cb20f2..f19bd0e0407 100644 --- a/Documentation/config/rebase.txt +++ b/Documentation/config/rebase.txt @@ -21,6 +21,9 @@ rebase.autoStash:: `--autostash` options of linkgit:git-rebase[1]. Defaults to false. +rebase.updateRefs:: + If set to true enable `--update-refs` option by default. + rebase.missingCommitsCheck:: If set to "warn", git rebase -i will print a warning if some commits are removed (e.g. a line was deleted), however the diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index dfd847d6615..305255f1e15 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -614,6 +614,9 @@ start would be overridden by the presence of Automatically force-update any branches that point to commits that are being rebased. Any branches that are checked out in a worktree are not updated in this way. ++ +If the configuration variable `rebase.updateRefs` is set, then this option +can be used to override and disable this setting. INCOMPATIBLE OPTIONS -------------------- diff --git a/builtin/rebase.c b/builtin/rebase.c index fd64897132a..dbb91895687 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -802,6 +802,11 @@ static int rebase_config(const char *var, const char *value, void *data) return 0; } + if (!strcmp(var, "rebase.updaterefs")) { + opts->update_refs = git_config_bool(var, value); + return 0; + } + if (!strcmp(var, "rebase.reschedulefailedexec")) { opts->reschedule_failed_exec = git_config_bool(var, value); return 0; diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index bebf9b4def7..1a27bb0626d 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1772,6 +1772,12 @@ test_expect_success '--update-refs adds label and update-ref commands' ' EOF test_must_fail git rebase -i --autosquash --update-refs primary >todo && + test_cmp expect todo && + + test_must_fail git -c rebase.autosquash=true \ + -c rebase.updaterefs=true \ + rebase -i primary >todo && + test_cmp expect todo ) ' @@ -1813,6 +1819,14 @@ test_expect_success '--update-refs adds commands with --rebase-merges' ' --rebase-merges=rebase-cousins \ --update-refs primary >todo && + test_cmp expect todo && + + test_must_fail git -c rebase.autosquash=true \ + -c rebase.updaterefs=true \ + rebase -i \ + --rebase-merges=rebase-cousins \ + primary >todo && + test_cmp expect todo ) ' -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v5 11/12] sequencer: ignore HEAD ref under --update-refs 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (9 preceding siblings ...) 2022-07-19 18:33 ` [PATCH v5 10/12] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 12/12] sequencer: notify user of --update-refs activity Derrick Stolee via GitGitGadget ` (2 subsequent siblings) 13 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> When using the 'git rebase -i --update-refs' option, the todo list is populated with 'update-ref' commands for all tip refs in the history that is being rebased. Refs that are checked out by some worktree are instead added as a comment to warn the user that they will not be updated. Until now, this included the HEAD ref, which is being updated by the rebase process itself, regardless of the --update-refs option. Remove the comment in this case by ignoring any decorations that match the HEAD ref. Reported-by: Elijah Newren <newren@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/sequencer.c b/sequencer.c index 67812c0294f..1602649332b 100644 --- a/sequencer.c +++ b/sequencer.c @@ -5853,12 +5853,25 @@ static int add_decorations_to_list(const struct commit *commit, struct todo_add_branch_context *ctx) { const struct name_decoration *decoration = get_name_decoration(&commit->object); + const char *head_ref = resolve_ref_unsafe("HEAD", + RESOLVE_REF_READING, + NULL, + NULL); while (decoration) { struct todo_item *item; const char *path; size_t base_offset = ctx->buf->len; + /* + * If the branch is the current HEAD, then it will be + * updated by the default rebase behavior. + */ + if (head_ref && !strcmp(head_ref, decoration->name)) { + decoration = decoration->next; + continue; + } + ALLOC_GROW(ctx->items, ctx->items_nr + 1, ctx->items_alloc); -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH v5 12/12] sequencer: notify user of --update-refs activity 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (10 preceding siblings ...) 2022-07-19 18:33 ` [PATCH v5 11/12] sequencer: ignore HEAD ref under --update-refs Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 ` Derrick Stolee via GitGitGadget 2022-07-21 4:35 ` [PATCH v5 00/12] rebase: update branches in multi-part topic Elijah Newren 2022-07-21 14:04 ` Phillip Wood 13 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee via GitGitGadget @ 2022-07-19 18:33 UTC (permalink / raw) To: git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Phillip Wood, Elijah Newren, SZEDER Gábor, Derrick Stolee, Derrick Stolee From: Derrick Stolee <derrickstolee@github.com> When the user runs 'git rebase -i --update-refs', the end message still says only Successfully rebased and updated <HEAD-ref>. Update the sequencer to collect the successful (and unsuccessful) ref updates due to the --update-refs option, so the end message now says Successfully rebased and updated <HEAD-ref>. Updated the following refs with --update-refs: refs/heads/first refs/heads/third Failed to update the following refs with --update-refs: refs/heads/second To test this output, we need to be very careful to format the expected error to drop the leading tab characters. Also, we need to be aware that the verbose output from 'git rebase' is writing progress lines which don't use traditional newlines but clear the line after every progress item is complete. When opening the error file in an editor, these lines are visible, but when looking at the diff in a terminal those lines disappear because of the characters that delete the previous characters. Use 'sed' to clear those progress lines and clear the tabs so we can get an exact match on our expected output. Reported-by: Elijah Newren <newren@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> --- sequencer.c | 40 +++++++++++++++++++++++++++++------ t/t3404-rebase-interactive.sh | 37 ++++++++++++++++++++++++++++---- 2 files changed, 67 insertions(+), 10 deletions(-) diff --git a/sequencer.c b/sequencer.c index 1602649332b..96e1d58ede8 100644 --- a/sequencer.c +++ b/sequencer.c @@ -4261,26 +4261,54 @@ static int do_update_ref(struct repository *r, const char *refname) return 0; } -static int do_update_refs(struct repository *r) +static int do_update_refs(struct repository *r, int quiet) { int res = 0; struct string_list_item *item; struct string_list refs_to_oids = STRING_LIST_INIT_DUP; struct ref_store *refs = get_main_ref_store(r); + struct strbuf update_msg = STRBUF_INIT; + struct strbuf error_msg = STRBUF_INIT; if ((res = sequencer_get_update_refs_state(r->gitdir, &refs_to_oids))) return res; for_each_string_list_item(item, &refs_to_oids) { struct update_ref_record *rec = item->util; + int loop_res; - res |= refs_update_ref(refs, "rewritten during rebase", - item->string, - &rec->after, &rec->before, - 0, UPDATE_REFS_MSG_ON_ERR); + loop_res = refs_update_ref(refs, "rewritten during rebase", + item->string, + &rec->after, &rec->before, + 0, UPDATE_REFS_MSG_ON_ERR); + res |= loop_res; + + if (quiet) + continue; + + if (loop_res) + strbuf_addf(&error_msg, "\t%s\n", item->string); + else + strbuf_addf(&update_msg, "\t%s\n", item->string); + } + + if (!quiet && + (update_msg.len || error_msg.len)) { + fprintf(stderr, + _("Updated the following refs with %s:\n%s"), + "--update-refs", + update_msg.buf); + + if (res) + fprintf(stderr, + _("Failed to update the following refs with %s:\n%s"), + "--update-refs", + error_msg.buf); } string_list_clear(&refs_to_oids, 1); + strbuf_release(&update_msg); + strbuf_release(&error_msg); return res; } @@ -4800,7 +4828,7 @@ cleanup_head_ref: strbuf_release(&buf); strbuf_release(&head_ref); - if (do_update_refs(r)) + if (do_update_refs(r, opts->quiet)) return -1; } diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh index 1a27bb0626d..688b01e3eb6 100755 --- a/t/t3404-rebase-interactive.sh +++ b/t/t3404-rebase-interactive.sh @@ -1840,12 +1840,26 @@ test_expect_success '--update-refs updates refs correctly' ' test_commit extra2 fileX && git commit --amend --fixup=L && - git rebase -i --autosquash --update-refs primary && + git rebase -i --autosquash --update-refs primary 2>err && test_cmp_rev HEAD~3 refs/heads/first && test_cmp_rev HEAD~3 refs/heads/second && test_cmp_rev HEAD~1 refs/heads/third && - test_cmp_rev HEAD refs/heads/no-conflict-branch + test_cmp_rev HEAD refs/heads/no-conflict-branch && + + cat >expect <<-\EOF && + Successfully rebased and updated refs/heads/update-refs. + Updated the following refs with --update-refs: + refs/heads/first + refs/heads/no-conflict-branch + refs/heads/second + refs/heads/third + EOF + + # Clear "Rebasing (X/Y)" progress lines and drop leading tabs. + sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \ + <err >err.trimmed && + test_cmp expect err.trimmed ' test_expect_success 'respect user edits to update-ref steps' ' @@ -1983,8 +1997,23 @@ test_expect_success '--update-refs: check failed ref update' ' # the lock in the update-refs file. git rev-parse third >.git/refs/heads/second && - git rebase --continue 2>err && - grep "update_ref failed for ref '\''refs/heads/second'\''" err + test_must_fail git rebase --continue 2>err && + grep "update_ref failed for ref '\''refs/heads/second'\''" err && + + cat >expect <<-\EOF && + Updated the following refs with --update-refs: + refs/heads/first + refs/heads/no-conflict-branch + refs/heads/third + Failed to update the following refs with --update-refs: + refs/heads/second + EOF + + # Clear "Rebasing (X/Y)" progress lines and drop leading tabs. + tail -n 6 err >err.last && + sed -e "s/Rebasing.*Successfully/Successfully/g" -e "s/^\t//g" \ + <err.last >err.trimmed && + test_cmp expect err.trimmed ' # This must be the last test in this file -- gitgitgadget ^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH v5 00/12] rebase: update branches in multi-part topic 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (11 preceding siblings ...) 2022-07-19 18:33 ` [PATCH v5 12/12] sequencer: notify user of --update-refs activity Derrick Stolee via GitGitGadget @ 2022-07-21 4:35 ` Elijah Newren 2022-07-21 12:12 ` Derrick Stolee 2022-07-21 14:04 ` Phillip Wood 13 siblings, 1 reply; 144+ messages in thread From: Elijah Newren @ 2022-07-21 4:35 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood, SZEDER Gábor, Derrick Stolee On Tue, Jul 19, 2022 at 11:33 AM Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com> wrote: > > This series is based on ds/branch-checked-out. > > This is a feature I've wanted for quite a while. When working on the sparse > index topic, I created a long RFC that actually broke into three topics for > full review upstream. These topics were sequential, so any feedback on an > earlier one required updates to the later ones. I would work on the full > feature and use interactive rebase to update the full list of commits. > However, I would need to update the branches pointing to those sub-topics. > > This series adds a new --update-refs option to 'git rebase' (along with a > rebase.updateRefs config option) that adds 'update-ref' commands into the > TODO list. This is powered by the commit decoration machinery. > > As an example, here is my in-progress bundle URI RFC split into subtopics as > they appear during the TODO list of a git rebase -i --update-refs: > > pick 2d966282ff3 docs: document bundle URI standard > pick 31396e9171a remote-curl: add 'get' capability > pick 54c6ab70f67 bundle-uri: create basic file-copy logic > pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// > pick 6adaf842684 fetch: add --bundle-uri option > pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration > update-ref refs/heads/bundle-redo/fetch > > pick 1e3f6546632 clone: add --bundle-uri option > pick 9e4a6fe9b68 clone: --bundle-uri cannot be combined with --depth > update-ref refs/heads/bundle-redo/clone > > pick 5451cb6599c bundle-uri: create bundle_list struct and helpers > pick 3029c3aca15 bundle-uri: create base key-value pair parsing > pick a8b2de79ce8 bundle-uri: create "key=value" line parsing > pick 92625a47673 bundle-uri: unit test "key=value" parsing > pick a8616af4dc2 bundle-uri: limit recursion depth for bundle lists > pick 9d6809a8d53 bundle-uri: parse bundle list in config format > pick 287a732b54c bundle-uri: fetch a list of bundles > update-ref refs/heads/bundle-redo/list > > pick b09f8226185 protocol v2: add server-side "bundle-uri" skeleton > pick 520204dcd1c bundle-uri client: add minimal NOOP client > pick 62e8b457b48 bundle-uri client: add "git ls-remote-bundle-uri" > pick 00eae925043 bundle-uri: serve URI advertisement from bundle.* config > pick 4277440a250 bundle-uri client: add boolean transfer.bundleURI setting > pick caf4599a81d bundle-uri: allow relative URLs in bundle lists > pick df255000b7e bundle-uri: download bundles from an advertised list > pick d71beabf199 clone: unbundle the advertised bundles > pick c9578391976 t5601: basic bundle URI tests > # Ref refs/heads/bundle-redo/rfc-3 checked out at '/home/stolee/_git/git-bundles' > > update-ref refs/heads/bundle-redo/advertise > > > Here is an outline of the series: > > * Patch 1 updates some tests for branch_checked_out() to use 'git bisect' > and 'git rebase' as black-boxes instead of manually editing files inside > $GIT_DIR. (Thanks, Junio!) > * Patch 2 updates some tests for branch_checked_out() for the 'apply' > backend. > * Patch 3 updates branch_checked_out() to parse the > rebase-merge/update-refs file to block concurrent ref updates and > checkouts on branches selected by --update-refs. > * Patch 4 updates the todo list documentation to remove some unnecessary > dots in the 'merge' command. This makes it consistent with the 'fixup' > command before we document the 'update-ref' command. > * Patch 5 updates the definition of todo_command_info to use enum values as > array indices. > * Patches 6-8 implement the --update-refs logic itself. > * Patch 9 specifically updates the update-refs file every time the user > edits the todo-list (Thanks Phillip!) > * Patch 10 adds the rebase.updateRefs config option similar to > rebase.autoSquash. > * Patch 11 ignores the HEAD ref when creating the todo list instead of > making a comment (Thanks Elijah!) > * Patch 12 adds messaging to the end of the rebase stating which refs were > updated (Thanks Elijah!) > > During review, we have identified some areas that would be good for > #leftoverbits: > > * Warn the user when they add an 'update-ref ' command but is checked out > in another worktree. > * The checks in patch 9 are quadratic. They could be sped up using > hashtables. > * Consider whether we should include an 'update-ref ' command for the HEAD > ref, so that all refs are updated in the same way. This might help > confused users. Not necessarily so they are updated in the same way; the behind the scenes mechanism could perhaps still be different. Just so that if the user looks for the "list of things being updated" they don't get surprised that HEAD is missing. > * The error message for failed ref updates could include information on the > commit IDs that would have been used. This can help the user fix the > situation by updating the refs manually. > * Modify the --update-refs option from a boolean to an > optionally-string-parameter that specifies refspecs for the 'update-ref' > commands. refspecs? Is that the term you really mean here? > Updates in v5 > ============= > > * Rename 'wt_dir' to 'wt_git_dir' for clarity. > * The documented behavior around 'fixup!' and 'squash!' commits was > incorrect, so update the commit message, documentation, and test to > demonstrate the actual behavior. > * Use CALLOC_ARRAY() to be more idiomatic. > * Be much more careful about propagating errors. > * Commit message typo: "We an" to "We can" > * Remove unnecessary null OID check when writing refs, since those would > already be removed by a previous step. Thanks, I've read over the range-diff and these changes look good to me. One thing I'm curious about (sorry to bring this up so late): "pick" commands come with the old commit hash. Perhaps the update-ref commands should too? (e.g. "update-ref refs/heads/topic from <OLDHASH>") ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v5 00/12] rebase: update branches in multi-part topic 2022-07-21 4:35 ` [PATCH v5 00/12] rebase: update branches in multi-part topic Elijah Newren @ 2022-07-21 12:12 ` Derrick Stolee 2022-07-21 19:43 ` Elijah Newren 0 siblings, 1 reply; 144+ messages in thread From: Derrick Stolee @ 2022-07-21 12:12 UTC (permalink / raw) To: Elijah Newren, Derrick Stolee via GitGitGadget Cc: Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood, SZEDER Gábor On 7/21/2022 12:35 AM, Elijah Newren wrote: > On Tue, Jul 19, 2022 at 11:33 AM Derrick Stolee via GitGitGadget >> During review, we have identified some areas that would be good for >> #leftoverbits: >> >> * Warn the user when they add an 'update-ref ' command but is checked out >> in another worktree. >> * The checks in patch 9 are quadratic. They could be sped up using >> hashtables. >> * Consider whether we should include an 'update-ref ' command for the HEAD >> ref, so that all refs are updated in the same way. This might help >> confused users. > > Not necessarily so they are updated in the same way; the behind the > scenes mechanism could perhaps still be different. Just so that if > the user looks for the "list of things being updated" they don't get > surprised that HEAD is missing. Noted. >> * The error message for failed ref updates could include information on the >> commit IDs that would have been used. This can help the user fix the >> situation by updating the refs manually. >> * Modify the --update-refs option from a boolean to an >> optionally-string-parameter that specifies refspecs for the 'update-ref' >> commands. > > refspecs? Is that the term you really mean here? Probably "ref namespaces" or "ref prefixes" would be better. >> Updates in v5 >> ============= >> >> * Rename 'wt_dir' to 'wt_git_dir' for clarity. >> * The documented behavior around 'fixup!' and 'squash!' commits was >> incorrect, so update the commit message, documentation, and test to >> demonstrate the actual behavior. >> * Use CALLOC_ARRAY() to be more idiomatic. >> * Be much more careful about propagating errors. >> * Commit message typo: "We an" to "We can" >> * Remove unnecessary null OID check when writing refs, since those would >> already be removed by a previous step. > > Thanks, I've read over the range-diff and these changes look good to > me. One thing I'm curious about (sorry to bring this up so late): > "pick" commands come with the old commit hash. Perhaps the update-ref > commands should too? (e.g. "update-ref refs/heads/topic from > <OLDHASH>") I don't personally see the value here other than to make it harder for someone to add new commands (and confusing when wanting to create a brand new ref). We could consider adding a comment in the future without any backwards compatibility issues: update-ref refs/heads/my-ref # was 0123dead Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v5 00/12] rebase: update branches in multi-part topic 2022-07-21 12:12 ` Derrick Stolee @ 2022-07-21 19:43 ` Elijah Newren 2022-07-21 20:05 ` Derrick Stolee 0 siblings, 1 reply; 144+ messages in thread From: Elijah Newren @ 2022-07-21 19:43 UTC (permalink / raw) To: Derrick Stolee Cc: Derrick Stolee via GitGitGadget, Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood, SZEDER Gábor On Thu, Jul 21, 2022 at 5:13 AM Derrick Stolee <derrickstolee@github.com> wrote: > > On 7/21/2022 12:35 AM, Elijah Newren wrote: > > > One thing I'm curious about (sorry to bring this up so late): > > "pick" commands come with the old commit hash. Perhaps the update-ref > > commands should too? (e.g. "update-ref refs/heads/topic from > > <OLDHASH>") > > I don't personally see the value here other than to make it harder > for someone to add new commands (and confusing when wanting to > create a brand new ref). I apologize if my late query caused frustration. I was thinking it made more sense to ask now than later and was genuinely curious for thoughts on the idea, but perhaps it'd have been better if I just didn't bring it up. Sorry. But, if you're curious why I was thinking about it... There are four ways I can think of to handle the <oldvalue>: (A) include it in the "update-ref" instruction, (B) manually get the <oldvalue> at program invocation, (C) accept an increased race-window by not grabbing <oldvalue> until the line is parsed, or (D) ignore races and just don't provide <oldvalue> when updating refs. If one chooses (B), they can either (B1) pre-parse the entire todo list and look up the current values of any refs mentioned in an update-ref instruction, or (B2) record the value of all existing refs at invocation. Rebase is already paying the cost for pre-parsing the entire todo list (i.e. B1). In fact, you _also_ preemptively locking the refs mentioned. So, in a sense, rebase is already covered. However: * adding such a thing could potentially remove a small race window (perhaps the user takes a while to edit the todo list, and the ref somehow gets updated before they even finish that step; being able to specify the oldref might help in such a case). * existing commands (pick, merge, etc.) give the oldrefs (or labels to them), so having update-ref also do so might provide some "feel good" consistency. Perhaps those reasons are pretty weak for the rebase case, given you already do B1. However, `git replay` isn't going to do B1, and nor will it preemptively lock refs. I see `git replay` as a patch-based analogue to the snapshot-based tools of fast-export and fast-import. (fast-import doesn't pre-parse its entire input before starting to process the first commit, nor does it preemptively lock refs.) B2 is not appealing because some repositories have gargantuan numbers of refs. Also, the update-ref-during-edit-of-todo rationale is more important for `git replay`, because I want `git replay` to accept a filename containing a todo list in lieu of a range of revisions to replay -- perhaps users spent quite some time or programming effort generating their todo list and it took some time to generate/review. And that further undercuts both B1 and B2. So, between all those reasons, (A) is very appealing to me. All that said, `git replay` already has other planned differences in its todo list (e.g. a "reset" directive must be the first non-comment directive, there's no separate "merge" directive, etc.), so if I have to add another one, it's no big deal. Your point about specifying <oldref> being seen as friction for some users is well taken, and I'm thinking of making the "from <oldref>" part of the instruction optional (though documenting that users are accepting some race risk by switching from (A) to (C) when they omit it). > We could consider adding a comment in the > future without any backwards compatibility issues: > > update-ref refs/heads/my-ref # was 0123dead Yes, this could certainly be added later (though none of my reasons for wanting <oldref> were geared around pointers for the end user, so this form wouldn't be helpful to me). Also, if "from <oldref>" is optional, it could also be added later. If you've read this far, let me just take a minute to again thank you for your work on this --update-refs option to rebase. It's some cool work. Also, thanks for listening to my random ideas and musings along the way. :-) ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v5 00/12] rebase: update branches in multi-part topic 2022-07-21 19:43 ` Elijah Newren @ 2022-07-21 20:05 ` Derrick Stolee 0 siblings, 0 replies; 144+ messages in thread From: Derrick Stolee @ 2022-07-21 20:05 UTC (permalink / raw) To: Elijah Newren Cc: Derrick Stolee via GitGitGadget, Git Mailing List, Junio C Hamano, Johannes Schindelin, Taylor Blau, Jeff Hostetler, Phillip Wood, SZEDER Gábor On 7/21/2022 3:43 PM, Elijah Newren wrote: > On Thu, Jul 21, 2022 at 5:13 AM Derrick Stolee <derrickstolee@github.com> wrote: >> >> On 7/21/2022 12:35 AM, Elijah Newren wrote: >> >>> One thing I'm curious about (sorry to bring this up so late): >>> "pick" commands come with the old commit hash. Perhaps the update-ref >>> commands should too? (e.g. "update-ref refs/heads/topic from >>> <OLDHASH>") >> >> I don't personally see the value here other than to make it harder >> for someone to add new commands (and confusing when wanting to >> create a brand new ref). > > I apologize if my late query caused frustration. I intended this as "I don't understand, please enlighten me", but looking back now it definitely sounds more frustrated. Sorry about that. > I was thinking it > made more sense to ask now than later and was genuinely curious for > thoughts on the idea, but perhaps it'd have been better if I just > didn't bring it up. Sorry. It's definitely the time to bring it up, because it will be harder to change it once it's shipped. > But, if you're curious why I was thinking about it... > > > There are four ways I can think of to handle the <oldvalue>: (A) > include it in the "update-ref" instruction, (B) manually get the > <oldvalue> at program invocation, (C) accept an increased race-window > by not grabbing <oldvalue> until the line is parsed, or (D) ignore > races and just don't provide <oldvalue> when updating refs. If one > chooses (B), they can either (B1) pre-parse the entire todo list and > look up the current values of any refs mentioned in an update-ref > instruction, or (B2) record the value of all existing refs at > invocation. > > Rebase is already paying the cost for pre-parsing the entire todo list > (i.e. B1). In fact, you _also_ preemptively locking the refs > mentioned. So, in a sense, rebase is already covered. However: > * adding such a thing could potentially remove a small race window > (perhaps the user takes a while to edit the todo list, and the ref > somehow gets updated before they even finish that step; being able to > specify the oldref might help in such a case). > * existing commands (pick, merge, etc.) give the oldrefs (or labels > to them), so having update-ref also do so might provide some "feel > good" consistency. > > Perhaps those reasons are pretty weak for the rebase case, given you > already do B1. However, `git replay` isn't going to do B1, and nor > will it preemptively lock refs. I see `git replay` as a patch-based > analogue to the snapshot-based tools of fast-export and fast-import. > (fast-import doesn't pre-parse its entire input before starting to > process the first commit, nor does it preemptively lock refs.) B2 is > not appealing because some repositories have gargantuan numbers of > refs. Also, the update-ref-during-edit-of-todo rationale is more > important for `git replay`, because I want `git replay` to accept a > filename containing a todo list in lieu of a range of revisions to > replay -- perhaps users spent quite some time or programming effort > generating their todo list and it took some time to generate/review. > And that further undercuts both B1 and B2. So, between all those > reasons, (A) is very appealing to me. > > All that said, `git replay` already has other planned differences in > its todo list (e.g. a "reset" directive must be the first non-comment > directive, there's no separate "merge" directive, etc.), so if I have > to add another one, it's no big deal. I'm happy that you're working on 'git replay'. I'm excited to see how it works. Also, it would be helpful to keep these things in line as much as possible. > Your point about specifying <oldref> being seen as friction for some > users is well taken, and I'm thinking of making the "from <oldref>" > part of the instruction optional (though documenting that users are > accepting some race risk by switching from (A) to (C) when they omit > it). > >> We could consider adding a comment in the >> future without any backwards compatibility issues: >> >> update-ref refs/heads/my-ref # was 0123dead > > Yes, this could certainly be added later (though none of my reasons > for wanting <oldref> were geared around pointers for the end user, so > this form wouldn't be helpful to me). > > Also, if "from <oldref>" is optional, it could also be added later. The optional value makes this something that we can add later, which I think is good flexibility. I was thinking about doing the comment thing as a follow-up (in addition to Phillip's error mode comments) but maybe I'll explore this direction instead. > If you've read this far, let me just take a minute to again thank you > for your work on this --update-refs option to rebase. It's some cool > work. Also, thanks for listening to my random ideas and musings along > the way. :-) I really appreciate the attention you've given it. It's already improved a lot due to your help. Thanks, -Stolee ^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH v5 00/12] rebase: update branches in multi-part topic 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget ` (12 preceding siblings ...) 2022-07-21 4:35 ` [PATCH v5 00/12] rebase: update branches in multi-part topic Elijah Newren @ 2022-07-21 14:04 ` Phillip Wood 13 siblings, 0 replies; 144+ messages in thread From: Phillip Wood @ 2022-07-21 14:04 UTC (permalink / raw) To: Derrick Stolee via GitGitGadget, git Cc: gitster, johannes.schindelin, me, Jeff Hostetler, Elijah Newren, SZEDER Gábor, Derrick Stolee Hi Stolee I'm afraid I've only had time to have a quick look at patches 8 & 9. The error propagation is looking better but I did notice a couple of calls to write_update_refs_state() where we don't propagate the return value. Best Wishes Phillip On 19/07/2022 19:33, Derrick Stolee via GitGitGadget wrote: > This series is based on ds/branch-checked-out. > > This is a feature I've wanted for quite a while. When working on the sparse > index topic, I created a long RFC that actually broke into three topics for > full review upstream. These topics were sequential, so any feedback on an > earlier one required updates to the later ones. I would work on the full > feature and use interactive rebase to update the full list of commits. > However, I would need to update the branches pointing to those sub-topics. > > This series adds a new --update-refs option to 'git rebase' (along with a > rebase.updateRefs config option) that adds 'update-ref' commands into the > TODO list. This is powered by the commit decoration machinery. > > As an example, here is my in-progress bundle URI RFC split into subtopics as > they appear during the TODO list of a git rebase -i --update-refs: > > pick 2d966282ff3 docs: document bundle URI standard > pick 31396e9171a remote-curl: add 'get' capability > pick 54c6ab70f67 bundle-uri: create basic file-copy logic > pick 96cb2e35af1 bundle-uri: add support for http(s):// and file:// > pick 6adaf842684 fetch: add --bundle-uri option > pick 6c5840ed77e fetch: add 'refs/bundle/' to log.excludeDecoration > update-ref refs/heads/bundle-redo/fetch > > pick 1e3f6546632 clone: add --bundle-uri option > pick 9e4a6fe9b68 clone: --bundle-uri cannot be combined with --depth > update-ref refs/heads/bundle-redo/clone > > pick 5451cb6599c bundle-uri: create bundle_list struct and helpers > pick 3029c3aca15 bundle-uri: create base key-value pair parsing > pick a8b2de79ce8 bundle-uri: create "key=value" line parsing > pick 92625a47673 bundle-uri: unit test "key=value" parsing > pick a8616af4dc2 bundle-uri: limit recursion depth for bundle lists > pick 9d6809a8d53 bundle-uri: parse bundle list in config format > pick 287a732b54c bundle-uri: fetch a list of bundles > update-ref refs/heads/bundle-redo/list > > pick b09f8226185 protocol v2: add server-side "bundle-uri" skeleton > pick 520204dcd1c bundle-uri client: add minimal NOOP client > pick 62e8b457b48 bundle-uri client: add "git ls-remote-bundle-uri" > pick 00eae925043 bundle-uri: serve URI advertisement from bundle.* config > pick 4277440a250 bundle-uri client: add boolean transfer.bundleURI setting > pick caf4599a81d bundle-uri: allow relative URLs in bundle lists > pick df255000b7e bundle-uri: download bundles from an advertised list > pick d71beabf199 clone: unbundle the advertised bundles > pick c9578391976 t5601: basic bundle URI tests > # Ref refs/heads/bundle-redo/rfc-3 checked out at '/home/stolee/_git/git-bundles' > > update-ref refs/heads/bundle-redo/advertise > > > Here is an outline of the series: > > * Patch 1 updates some tests for branch_checked_out() to use 'git bisect' > and 'git rebase' as black-boxes instead of manually editing files inside > $GIT_DIR. (Thanks, Junio!) > * Patch 2 updates some tests for branch_checked_out() for the 'apply' > backend. > * Patch 3 updates branch_checked_out() to parse the > rebase-merge/update-refs file to block concurrent ref updates and > checkouts on branches selected by --update-refs. > * Patch 4 updates the todo list documentation to remove some unnecessary > dots in the 'merge' command. This makes it consistent with the 'fixup' > command before we document the 'update-ref' command. > * Patch 5 updates the definition of todo_command_info to use enum values as > array indices. > * Patches 6-8 implement the --update-refs logic itself. > * Patch 9 specifically updates the update-refs file every time the user > edits the todo-list (Thanks Phillip!) > * Patch 10 adds the rebase.updateRefs config option similar to > rebase.autoSquash. > * Patch 11 ignores the HEAD ref when creating the todo list instead of > making a comment (Thanks Elijah!) > * Patch 12 adds messaging to the end of the rebase stating which refs were > updated (Thanks Elijah!) > > During review, we have identified some areas that would be good for > #leftoverbits: > > * Warn the user when they add an 'update-ref ' command but is checked out > in another worktree. > * The checks in patch 9 are quadratic. They could be sped up using > hashtables. > * Consider whether we should include an 'update-ref ' command for the HEAD > ref, so that all refs are updated in the same way. This might help > confused users. > * The error message for failed ref updates could include information on the > commit IDs that would have been used. This can help the user fix the > situation by updating the refs manually. > * Modify the --update-refs option from a boolean to an > optionally-string-parameter that specifies refspecs for the 'update-ref' > commands. > > > Updates in v5 > ============= > > * Rename 'wt_dir' to 'wt_git_dir' for clarity. > * The documented behavior around 'fixup!' and 'squash!' commits was > incorrect, so update the commit message, documentation, and test to > demonstrate the actual behavior. > * Use CALLOC_ARRAY() to be more idiomatic. > * Be much more careful about propagating errors. > * Commit message typo: "We an" to "We can" > * Remove unnecessary null OID check when writing refs, since those would > already be removed by a previous step. > > > Updates in v4 > ============= > > This version took longer than I'd hoped (I had less time to work on it than > anticipated) but it also has some major updates. These major updates are > direct responses to the significant review this series has received. Thank > you! > > * The update-refs file now stores "ref/before/after" triples (still > separated by lines). This allows us to store the "before" OID of a ref in > addition to the "after" that we will write to that ref at the end of the > rebase. This allows us to do a "force-with-lease" update. The > branch_checked_out() updates should prevent Git from updating those refs > while under the rebase, but older versions and third-party tools don't > have that protection. > * The update-refs file is updated with every update to the todo-list file. > This allows for some advanced changes to the file, including removing, > adding, and duplicating 'update-ref' commands. > * The message at the end of the rebase process now lists which refs were > updated with the update-ref steps. This includes any ref updates that > fail. > * The branch_checked_out() tests now use 'git bisect' and 'git rebase' as > black-boxes instead of testing their internals directly. > > Here are the more minor updates: > > * Dropped an unnecessary stat() call. > * Updated commit messages to include extra details, based on confusion in > last round. > * The HEAD branch no longer appears as a comment line in the initial todo > list. > * The update-refs file is now written using a lockfile. > * Tests now use test_cmp_rev. > * A memory leak ('path' variable) is resolved. > > > Updates in v3 > ============= > > * The branch_checked_out() API was extracted to its own topic and is now > the ds/branch-checked-out branch. This series is now based on that one. > * The for_each_decoration() API was removed, since it became trivial once > it did not take a commit directly. > * The branch_checked_out() tests did not verify the rebase-apply data (for > the apply backend), so that is fixed. > * Instead of using the 'label' command and a final 'update-refs' command in > the todo list, use a new 'update-ref ' command. This command updates the > rebase-merge/update-refs file with the OID of HEAD at these steps. At the > very end of the rebase sequence, those refs are updated to the stored OID > values (assuming that they were not removed by the user, in which case we > notice that the OID is the null OID and we do nothing). > * New tests are added. > * The todo-list comment documentation has some new formatting updates, but > also includes a description of 'update-refs' in this version. > > > Updates in v2 > ============= > > As recommended by the excellent feedback, I have removed the 'exec' commands > in favor of the 'label' commands and a new 'update-refs' command at the very > end. This way, there is only one step that updates all of the refs at the > end instead of updating refs during the rebase. If a user runs 'git rebase > --abort' in the middle, then their refs are still where they need to be. > > Based on some of the discussion, it seemed like one way to do this would be > to have an 'update-ref ' command that would take the place of these 'label' > commands. However, this would require two things that make it a bit awkward: > > 1. We would need to replicate the storage of those positions during the > rebase. 'label' already does this pretty well. I've added the > "for-update-refs/" label to help here. > 2. If we want to close out all of the refs as the rebase is finishing, then > that "step" becomes invisible to the user (and a bit more complicated to > insert). Thus, the 'update-refs' step performs this action. If the user > wants to do things after that step, then they can do so by editing the > TODO list. > > Other updates: > > * The 'keep_decorations' parameter was renamed to 'update_refs'. > * I added tests for --rebase-merges=rebase-cousins to show how these labels > interact with other labels and merge commands. > * I changed the order of the insertion of these update-refs labels to be > before the fixups are rearranged. This fixes a bug where the tip commit > is a fixup! so its decorations are never inspected (and they would be in > the wrong place even if they were). The fixup! commands are properly > inserted between a pick and its following label command. Tests > demonstrate this is correct. > * Numerous style choices are updated based on feedback. > > Thank you for all of the detailed review and ideas in this space. I > appreciate any more ideas that can make this feature as effective as it can > be. > > Thanks, -Stolee > > Derrick Stolee (12): > t2407: test bisect and rebase as black-boxes > t2407: test branches currently using apply backend > branch: consider refs under 'update-refs' > rebase-interactive: update 'merge' description > sequencer: define array with enum values > sequencer: add update-ref command > rebase: add --update-refs option > rebase: update refs from 'update-ref' commands > sequencer: rewrite update-refs as user edits todo list > rebase: add rebase.updateRefs config option > sequencer: ignore HEAD ref under --update-refs > sequencer: notify user of --update-refs activity > > Documentation/config/rebase.txt | 3 + > Documentation/git-rebase.txt | 10 + > branch.c | 13 + > builtin/rebase.c | 10 + > rebase-interactive.c | 15 +- > sequencer.c | 474 +++++++++++++++++++++++++++++++- > sequencer.h | 23 ++ > t/lib-rebase.sh | 15 + > t/t2407-worktree-heads.sh | 103 +++++-- > t/t3404-rebase-interactive.sh | 273 ++++++++++++++++++ > 10 files changed, 895 insertions(+), 44 deletions(-) > > > base-commit: 9bef0b1e6ec371e786c2fba3edcc06ad040a536c > Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1247%2Fderrickstolee%2Frebase-keep-decorations-v5 > Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1247/derrickstolee/rebase-keep-decorations-v5 > Pull-Request: https://github.com/gitgitgadget/git/pull/1247 > > Range-diff vs v4: > > 1: 9e53a27017a = 1: 9e53a27017a t2407: test bisect and rebase as black-boxes > 2: 540a3be256f = 2: 540a3be256f t2407: test branches currently using apply backend > 3: bf301a054e3 ! 3: 1089a0edb73 branch: consider refs under 'update-refs' > @@ sequencer.c: static GIT_PATH_FUNC(rebase_path_squash_onto, "rebase-merge/squash- > + * rebase_path_update_refs() returns the path to this file for a given > + * worktree directory. For the current worktree, pass the_repository->gitdir. > + */ > -+static char *rebase_path_update_refs(const char *wt_dir) > ++static char *rebase_path_update_refs(const char *wt_git_dir) > +{ > -+ return xstrfmt("%s/rebase-merge/update-refs", wt_dir); > ++ return xstrfmt("%s/rebase-merge/update-refs", wt_git_dir); > +} > + > /* > 4: dec95681d2b = 4: d1cce4f06aa rebase-interactive: update 'merge' description > 5: b2c09600918 = 5: 4c086d477f0 sequencer: define array with enum values > 6: fa7ecb718cf = 6: 7b3d6601960 sequencer: add update-ref command > 7: 3ec2cc922f9 ! 7: 7efb55e4f14 rebase: add --update-refs option > @@ Commit message > --fixup commit at the tip of the feature to apply correctly to the sub > branch, even if it is fixing up the most-recent commit in that part. > > - One potential problem here is that refs decorating commits that are > - already marked as "fixup!" or "squash!" will not be included in this > - list. Generally, the reordering of the "fixup!" and "squash!" is likely > - to change the relative order of these refs, so it is not recommended. > - The workflow here is intended to allow these kinds of commits at the tip > - of the rebased branch while the other sub branches come along for the > - ride without intervention. > - > This change update the documentation and builtin to accept the > --update-refs option as well as updating the todo file with the > 'update-ref' commands. Tests are added to ensure that these todo > @@ Documentation/git-rebase.txt: provided. Otherwise an explicit `--no-reschedule-f > +--no-update-refs:: > + Automatically force-update any branches that point to commits that > + are being rebased. Any branches that are checked out in a worktree > -+ or point to a `squash! ...` or `fixup! ...` commit are not updated > -+ in this way. > ++ are not updated in this way. > + > INCOMPATIBLE OPTIONS > -------------------- > @@ builtin/rebase.c: int cmd_rebase(int argc, const char **argv, const char *prefix > N_("move commits that begin with " > "squash!/fixup! under -i")), > + OPT_BOOL(0, "update-refs", &options.update_refs, > -+ N_("update local refs that point to commits " > ++ N_("update branches that point to commits " > + "that are being rebased")), > { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"), > N_("GPG-sign commits"), > @@ t/t3404-rebase-interactive.sh: test_expect_success 'ORIG_HEAD is updated correct > + git branch -f second HEAD~3 && > + git branch -f third HEAD~1 && > + git commit --allow-empty --fixup=third && > ++ git branch -f is-not-reordered && > ++ git commit --allow-empty --fixup=HEAD~4 && > + git branch -f shared-tip && > + ( > + set_cat_todo_editor && > + > + cat >expect <<-EOF && > + pick $(git log -1 --format=%h J) J > ++ fixup $(git log -1 --format=%h update-refs) fixup! J # empty > + update-ref refs/heads/second > + update-ref refs/heads/first > + pick $(git log -1 --format=%h K) K > + pick $(git log -1 --format=%h L) L > -+ fixup $(git log -1 --format=%h update-refs) fixup! L # empty > ++ fixup $(git log -1 --format=%h is-not-reordered) fixup! L # empty > + update-ref refs/heads/third > + pick $(git log -1 --format=%h M) M > + update-ref refs/heads/no-conflict-branch > ++ update-ref refs/heads/is-not-reordered > + update-ref refs/heads/shared-tip > + EOF > + > 8: fb5f64c5201 ! 8: e7a91bdffbd rebase: update refs from 'update-ref' commands > @@ sequencer.c: struct update_ref_record { > > +static struct update_ref_record *init_update_ref_record(const char *ref) > +{ > -+ struct update_ref_record *rec = xmalloc(sizeof(*rec)); > ++ struct update_ref_record *rec; > ++ > ++ CALLOC_ARRAY(rec, 1); > + > + oidcpy(&rec->before, null_oid()); > + oidcpy(&rec->after, null_oid()); > @@ sequencer.c: leave_merge: > > -static int do_update_ref(struct repository *r, const char *ref_name) > +static int write_update_refs_state(struct string_list *refs_to_oids) > - { > ++{ > + int result = 0; > + struct lock_file lock = LOCK_INIT; > + FILE *fp = NULL; > @@ sequencer.c: leave_merge: > +} > + > +static int do_update_ref(struct repository *r, const char *refname) > -+{ > + { > + struct string_list_item *item; > + struct string_list list = STRING_LIST_INIT_DUP; > + > -+ sequencer_get_update_refs_state(r->gitdir, &list); > ++ if (sequencer_get_update_refs_state(r->gitdir, &list)) > ++ return -1; > + > + for_each_string_list_item(item, &list) { > + if (!strcmp(item->string, refname)) { > + struct update_ref_record *rec = item->util; > -+ read_ref("HEAD", &rec->after); > ++ if (read_ref("HEAD", &rec->after)) > ++ return -1; > + break; > + } > + } > @@ sequencer.c: leave_merge: > + struct string_list refs_to_oids = STRING_LIST_INIT_DUP; > + struct ref_store *refs = get_main_ref_store(r); > + > -+ sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); > ++ if ((res = sequencer_get_update_refs_state(r->gitdir, &refs_to_oids))) > ++ return res; > + > + for_each_string_list_item(item, &refs_to_oids) { > + struct update_ref_record *rec = item->util; > + > -+ if (oideq(&rec->after, the_hash_algo->null_oid)) { > -+ /* > -+ * Ref was not updated. User may have deleted the > -+ * 'update-ref' step. > -+ */ > -+ continue; > -+ } > -+ > + res |= refs_update_ref(refs, "rewritten during rebase", > + item->string, > + &rec->after, &rec->before, > @@ sequencer.c: leave_merge: > { > int i = todo_list->current; > @@ sequencer.c: cleanup_head_ref: > + > + strbuf_release(&buf); > strbuf_release(&head_ref); > ++ > ++ if (do_update_refs(r)) > ++ return -1; > } > > -+ do_update_refs(r); > -+ > /* > - * Sequence of picks finished successfully; cleanup by > - * removing the .git/sequencer directory > @@ sequencer.c: static int add_decorations_to_list(const struct commit *commit, > > sti = string_list_insert(&ctx->refs_to_oids, > @@ sequencer.c: static int add_decorations_to_list(const struct commit *commit, > } > > item->offset_in_buf = base_offset; > +@@ sequencer.c: static int add_decorations_to_list(const struct commit *commit, > + */ > + static int todo_list_add_update_ref_commands(struct todo_list *todo_list) > + { > +- int i; > ++ int i, res; > + static struct string_list decorate_refs_exclude = STRING_LIST_INIT_NODUP; > + static struct string_list decorate_refs_exclude_config = STRING_LIST_INIT_NODUP; > + static struct string_list decorate_refs_include = STRING_LIST_INIT_NODUP; > @@ sequencer.c: static int todo_list_add_update_ref_commands(struct todo_list *todo_list) > } > } > > -+ write_update_refs_state(&ctx.refs_to_oids); > ++ res = write_update_refs_state(&ctx.refs_to_oids); > + > string_list_clear(&ctx.refs_to_oids, 1); > ++ > ++ if (res) { > ++ /* we failed, so clean up the new list. */ > ++ free(ctx.items); > ++ return res; > ++ } > ++ > free(todo_list->items); > todo_list->items = ctx.items; > + todo_list->nr = ctx.items_nr; > > ## t/t2407-worktree-heads.sh ## > @@ t/t2407-worktree-heads.sh: test_expect_success !SANITIZE_LEAK 'refuse to overwrite: worktree in rebase (mer > 9: 29c7c76805a ! 9: 95e2bbcedb1 sequencer: rewrite update-refs as user edits todo list > @@ Commit message > > We can test that this works by rewriting the todo-list several times in > the course of a rebase. Check that each ref is locked or unlocked for > - updates after each todo-list update. We an also verify that the ref > + updates after each todo-list update. We can also verify that the ref > update fails if a concurrent process updates one of the refs after the > rebase process records the "locked" ref location. > > 10: c0022d07579 ! 10: a73b02568f3 rebase: add rebase.updateRefs config option > @@ Documentation/config/rebase.txt: rebase.autoStash:: > > ## Documentation/git-rebase.txt ## > @@ Documentation/git-rebase.txt: start would be overridden by the presence of > + Automatically force-update any branches that point to commits that > are being rebased. Any branches that are checked out in a worktree > - or point to a `squash! ...` or `fixup! ...` commit are not updated > - in this way. > + are not updated in this way. > ++ > +If the configuration variable `rebase.updateRefs` is set, then this option > +can be used to override and disable this setting. > 11: d53b4ff2cee = 11: 2a6577974c7 sequencer: ignore HEAD ref under --update-refs > 12: d5cd4b49e46 ! 12: ec080ce1e90 sequencer: notify user of --update-refs activity > @@ sequencer.c: static int do_update_ref(struct repository *r, const char *refname) > + struct strbuf update_msg = STRBUF_INIT; > + struct strbuf error_msg = STRBUF_INIT; > > - sequencer_get_update_refs_state(r->gitdir, &refs_to_oids); > + if ((res = sequencer_get_update_refs_state(r->gitdir, &refs_to_oids))) > + return res; > > for_each_string_list_item(item, &refs_to_oids) { > struct update_ref_record *rec = item->util; > + int loop_res; > > - if (oideq(&rec->after, the_hash_algo->null_oid)) { > - /* > -@@ sequencer.c: static int do_update_refs(struct repository *r) > - continue; > - } > - > - res |= refs_update_ref(refs, "rewritten during rebase", > - item->string, > - &rec->after, &rec->before, > @@ sequencer.c: static int do_update_refs(struct repository *r) > } > > @@ sequencer.c: cleanup_head_ref: > + strbuf_release(&buf); > strbuf_release(&head_ref); > - } > > -- do_update_refs(r); > -+ do_update_refs(r, opts->quiet); > +- if (do_update_refs(r)) > ++ if (do_update_refs(r, opts->quiet)) > + return -1; > + } > > - /* > - * Sequence of picks finished successfully; cleanup by > > ## t/t3404-rebase-interactive.sh ## > @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs updates refs correctly' ' > @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs updates refs c > > test_expect_success 'respect user edits to update-ref steps' ' > @@ t/t3404-rebase-interactive.sh: test_expect_success '--update-refs: check failed ref update' ' > + # the lock in the update-refs file. > git rev-parse third >.git/refs/heads/second && > > - git rebase --continue 2>err && > +- git rebase --continue 2>err && > - grep "update_ref failed for ref '\''refs/heads/second'\''" err > ++ test_must_fail git rebase --continue 2>err && > + grep "update_ref failed for ref '\''refs/heads/second'\''" err && > + > + cat >expect <<-\EOF && > ^ permalink raw reply [flat|nested] 144+ messages in thread
end of thread, other threads:[~2022-07-21 20:05 UTC | newest] Thread overview: 144+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-06-03 13:37 [PATCH 0/4] rebase: update branches in multi-part topic Derrick Stolee via GitGitGadget 2022-06-03 13:37 ` [PATCH 1/4] log-tree: create for_each_decoration() Derrick Stolee via GitGitGadget 2022-06-03 17:39 ` Junio C Hamano 2022-06-03 17:58 ` Derrick Stolee 2022-06-03 18:40 ` Junio C Hamano 2022-06-03 13:37 ` [PATCH 2/4] branch: add branch_checked_out() helper Derrick Stolee via GitGitGadget 2022-06-03 18:31 ` Junio C Hamano 2022-06-03 13:37 ` [PATCH 3/4] rebase: add --update-refs option Derrick Stolee via GitGitGadget 2022-06-07 10:25 ` Phillip Wood 2022-06-03 13:37 ` [PATCH 4/4] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget 2022-06-03 16:56 ` [PATCH 0/4] rebase: update branches in multi-part topic Junio C Hamano 2022-06-03 18:27 ` Taylor Blau 2022-06-03 18:52 ` Junio C Hamano 2022-06-03 19:59 ` Jeff Hostetler 2022-06-03 20:03 ` Taylor Blau 2022-06-03 21:23 ` Junio C Hamano 2022-06-04 15:28 ` Phillip Wood 2022-06-06 15:12 ` Derrick Stolee 2022-06-07 10:11 ` Phillip Wood 2022-06-07 19:39 ` Derrick Stolee 2022-06-08 16:03 ` Junio C Hamano 2022-06-06 16:36 ` Junio C Hamano 2022-06-07 6:25 ` Elijah Newren 2022-06-07 20:42 ` [PATCH v2 0/7] " Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 1/7] log-tree: create for_each_decoration() Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 2/7] branch: add branch_checked_out() helper Derrick Stolee via GitGitGadget 2022-06-07 22:09 ` Junio C Hamano 2022-06-08 2:14 ` Derrick Stolee 2022-06-08 2:43 ` Derrick Stolee 2022-06-08 4:52 ` Junio C Hamano 2022-06-07 20:42 ` [PATCH v2 3/7] sequencer: define array with enum values Derrick Stolee via GitGitGadget 2022-06-07 22:11 ` Junio C Hamano 2022-06-07 20:42 ` [PATCH v2 4/7] sequencer: add update-refs command Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 5/7] rebase: add --update-refs option Derrick Stolee via GitGitGadget 2022-06-07 20:42 ` [PATCH v2 6/7] sequencer: implement 'update-refs' command Derrick Stolee via GitGitGadget 2022-06-07 22:23 ` Junio C Hamano 2022-06-07 20:42 ` [PATCH v2 7/7] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget 2022-06-08 14:32 ` [PATCH v2 0/7] rebase: update branches in multi-part topic Phillip Wood 2022-06-08 18:09 ` Derrick Stolee 2022-06-09 10:04 ` Phillip Wood 2022-06-28 13:25 ` [PATCH v3 0/8] " Derrick Stolee via GitGitGadget 2022-06-28 13:25 ` [PATCH v3 1/8] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget 2022-06-28 20:44 ` Junio C Hamano 2022-06-29 12:54 ` Derrick Stolee 2022-06-30 16:44 ` Junio C Hamano 2022-06-30 17:35 ` Derrick Stolee 2022-06-28 13:25 ` [PATCH v3 2/8] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget 2022-06-28 20:48 ` Junio C Hamano 2022-06-29 12:58 ` Derrick Stolee 2022-06-30 9:47 ` Phillip Wood 2022-06-30 16:50 ` Junio C Hamano 2022-06-30 16:49 ` Junio C Hamano 2022-06-30 9:32 ` Phillip Wood 2022-06-30 13:35 ` Derrick Stolee 2022-07-01 13:40 ` Phillip Wood 2022-06-28 13:25 ` [PATCH v3 3/8] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget 2022-06-28 21:00 ` Junio C Hamano 2022-06-29 13:02 ` Derrick Stolee 2022-06-30 17:05 ` Junio C Hamano 2022-06-30 9:34 ` Phillip Wood 2022-06-28 13:25 ` [PATCH v3 4/8] sequencer: define array with enum values Derrick Stolee via GitGitGadget 2022-06-28 21:02 ` Junio C Hamano 2022-06-28 13:25 ` [PATCH v3 5/8] sequencer: add update-ref command Derrick Stolee via GitGitGadget 2022-06-30 9:39 ` Phillip Wood 2022-06-28 13:25 ` [PATCH v3 6/8] rebase: add --update-refs option Derrick Stolee via GitGitGadget 2022-06-28 21:09 ` Junio C Hamano 2022-06-29 13:03 ` Derrick Stolee 2022-07-01 9:20 ` Phillip Wood 2022-07-01 21:20 ` Elijah Newren 2022-07-04 12:56 ` Derrick Stolee 2022-07-04 17:57 ` Elijah Newren 2022-07-05 22:22 ` Derrick Stolee 2022-07-08 2:27 ` Elijah Newren 2022-06-28 13:25 ` [PATCH v3 7/8] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget 2022-06-28 21:15 ` Junio C Hamano 2022-06-29 13:05 ` Derrick Stolee 2022-06-30 17:09 ` Junio C Hamano 2022-06-29 13:06 ` Derrick Stolee 2022-07-01 9:31 ` Phillip Wood 2022-07-01 18:35 ` Junio C Hamano 2022-07-01 23:18 ` Elijah Newren 2022-07-04 13:05 ` Derrick Stolee 2022-06-28 13:25 ` [PATCH v3 8/8] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget 2022-06-28 21:19 ` [PATCH v3 0/8] rebase: update branches in multi-part topic Junio C Hamano 2022-07-01 13:43 ` Phillip Wood 2022-07-12 13:06 ` [PATCH v4 00/12] " Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 01/12] t2407: test bisect and rebase as black-boxes Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 02/12] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 03/12] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget 2022-07-15 15:37 ` Phillip Wood 2022-07-12 13:06 ` [PATCH v4 04/12] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 05/12] sequencer: define array with enum values Derrick Stolee via GitGitGadget 2022-07-12 13:06 ` [PATCH v4 06/12] sequencer: add update-ref command Derrick Stolee via GitGitGadget 2022-07-12 13:07 ` [PATCH v4 07/12] rebase: add --update-refs option Derrick Stolee via GitGitGadget 2022-07-16 19:30 ` Elijah Newren 2022-07-19 15:50 ` Derrick Stolee 2022-07-18 9:05 ` SZEDER Gábor 2022-07-18 16:55 ` Derrick Stolee 2022-07-18 19:35 ` Junio C Hamano 2022-07-19 15:53 ` Derrick Stolee 2022-07-19 16:44 ` Junio C Hamano 2022-07-19 16:47 ` Derrick Stolee 2022-07-12 13:07 ` [PATCH v4 08/12] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget 2022-07-15 13:25 ` Phillip Wood 2022-07-19 16:04 ` Derrick Stolee 2022-07-12 13:07 ` [PATCH v4 09/12] sequencer: rewrite update-refs as user edits todo list Derrick Stolee via GitGitGadget 2022-07-15 10:27 ` Phillip Wood 2022-07-15 13:13 ` Derrick Stolee 2022-07-18 13:09 ` Phillip Wood 2022-07-16 19:20 ` Elijah Newren 2022-07-12 13:07 ` [PATCH v4 10/12] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget 2022-07-12 13:07 ` [PATCH v4 11/12] sequencer: ignore HEAD ref under --update-refs Derrick Stolee via GitGitGadget 2022-07-12 13:07 ` [PATCH v4 12/12] sequencer: notify user of --update-refs activity Derrick Stolee via GitGitGadget 2022-07-15 10:12 ` Phillip Wood 2022-07-15 13:20 ` Derrick Stolee 2022-07-16 20:51 ` Elijah Newren 2022-07-16 22:09 ` Elijah Newren 2022-07-19 16:09 ` Derrick Stolee 2022-07-12 15:37 ` [PATCH v4 00/12] rebase: update branches in multi-part topic Junio C Hamano 2022-07-14 14:50 ` Derrick Stolee 2022-07-14 18:11 ` Junio C Hamano 2022-07-16 21:23 ` Elijah Newren 2022-07-16 20:56 ` Elijah Newren 2022-07-15 15:41 ` Phillip Wood 2022-07-19 18:33 ` [PATCH v5 " Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 01/12] t2407: test bisect and rebase as black-boxes Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 02/12] t2407: test branches currently using apply backend Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 03/12] branch: consider refs under 'update-refs' Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 04/12] rebase-interactive: update 'merge' description Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 05/12] sequencer: define array with enum values Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 06/12] sequencer: add update-ref command Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 07/12] rebase: add --update-refs option Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 08/12] rebase: update refs from 'update-ref' commands Derrick Stolee via GitGitGadget 2022-07-21 14:03 ` Phillip Wood 2022-07-19 18:33 ` [PATCH v5 09/12] sequencer: rewrite update-refs as user edits todo list Derrick Stolee via GitGitGadget 2022-07-21 14:04 ` Phillip Wood 2022-07-19 18:33 ` [PATCH v5 10/12] rebase: add rebase.updateRefs config option Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 11/12] sequencer: ignore HEAD ref under --update-refs Derrick Stolee via GitGitGadget 2022-07-19 18:33 ` [PATCH v5 12/12] sequencer: notify user of --update-refs activity Derrick Stolee via GitGitGadget 2022-07-21 4:35 ` [PATCH v5 00/12] rebase: update branches in multi-part topic Elijah Newren 2022-07-21 12:12 ` Derrick Stolee 2022-07-21 19:43 ` Elijah Newren 2022-07-21 20:05 ` Derrick Stolee 2022-07-21 14:04 ` Phillip Wood
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.