* [PATCH 0/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP @ 2021-07-29 12:27 ZheNing Hu via GitGitGadget 2021-07-29 12:27 ` [PATCH 1/2] " ZheNing Hu via GitGitGadget 2021-07-29 12:27 ` [PATCH 2/2] [GSOC] cherry-pick: use better advice message ZheNing Hu via GitGitGadget 0 siblings, 2 replies; 8+ messages in thread From: ZheNing Hu via GitGitGadget @ 2021-07-29 12:27 UTC (permalink / raw) To: git Cc: Junio C Hamano, Christian Couder, Hariom Verma, Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys, Ramkumar Ramachandra, Felipe Contreras, Phillip Wood, ZheNing Hu This patch fixes the bug when git cherry-pick is used with environment variable GIT_CHERRY_PICK_HELP, and makes git chery-pick advice message better. v2: https://lore.kernel.org/git/pull.1001.v2.git.1627135281887.gitgitgadget@gmail.com/ v2-->v3: 1. Add hidden option --rebase-preserve-merges-mode to git cherry-pick. 2. Split print_advice() into itself and check_need_delete_cherry_pick_head(). 3. Only allow git cherry-pick skip check_need_delete_cherry_pick_head(). 4. Use better git cherry-pick advice message. ZheNing Hu (2): [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP [GSOC] cherry-pick: use better advice message builtin/revert.c | 2 ++ git-rebase--preserve-merges.sh | 2 +- sequencer.c | 45 +++++++++++++++++++++++---------- sequencer.h | 1 + t/t3507-cherry-pick-conflict.sh | 44 +++++++++++++++++++++----------- 5 files changed, 64 insertions(+), 30 deletions(-) base-commit: daab8a564f8bbac55f70f8bf86c070e001a9b006 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1007%2Fadlternative%2Fcherry-pick-help-fix-2-v1 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1007/adlternative/cherry-pick-help-fix-2-v1 Pull-Request: https://github.com/gitgitgadget/git/pull/1007 -- gitgitgadget ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 1/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP 2021-07-29 12:27 [PATCH 0/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP ZheNing Hu via GitGitGadget @ 2021-07-29 12:27 ` ZheNing Hu via GitGitGadget 2021-07-29 20:56 ` Junio C Hamano 2021-07-29 12:27 ` [PATCH 2/2] [GSOC] cherry-pick: use better advice message ZheNing Hu via GitGitGadget 1 sibling, 1 reply; 8+ messages in thread From: ZheNing Hu via GitGitGadget @ 2021-07-29 12:27 UTC (permalink / raw) To: git Cc: Junio C Hamano, Christian Couder, Hariom Verma, Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys, Ramkumar Ramachandra, Felipe Contreras, Phillip Wood, ZheNing Hu, ZheNing Hu From: ZheNing Hu <adlternative@gmail.com> GIT_CHERRY_PICK_HELP is an environment variable, as the implementation detail of some porcelain in git to help realize the rebasing steps. E.g. `git rebase -p` set GIT_CHERRY_PICK_HELP value in git-rebase--preserve-merges.sh. But If we set the value of GIT_CHERRY_PICK_HELP when using `git cherry-pick`, CHERRY_PICK_HEAD will be deleted, then we will get an error when we try to use `git cherry-pick --continue` or other cherr-pick command. Introduce new "hidden" option --rebase-preserve-merges-mode for git cherry-pick which indicates that git cherry-pick is currently called by git-rebase--preserve-merges.sh. After `git rebase -p` completely abolished, this option should be removed. And then we split print_advice() into two part: check_need_delete_cherry_pick_head() check if we set GIT_CHERRY_PICK_HELP, if set, delete CHERRY_PICK_HEAD and return GIT_CHERRY_PICK_HELP's value, if not set, return NULL; The parameters of print_advice() have changed, which now accept a `struct replay_opts *opt` and a `const char *help_msgs`. We can pass the value of GIT_CHERRY_PICK_HELP into print_advice(), and output it. In this way, the steps of printing advice and checking GIT_CHERRY_PICK_HELP are decoupled. Finally, avoid deleting CHERRY_PICK_HEAD when we are truly cherry-picking, which can fix this breakage. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by Hariom Verma <hariom18599@gmail.com>: Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk> Hepled-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: ZheNing Hu <adlternative@gmail.com> --- builtin/revert.c | 2 ++ git-rebase--preserve-merges.sh | 2 +- sequencer.c | 36 ++++++++++++++++++++------------- sequencer.h | 1 + t/t3507-cherry-pick-conflict.sh | 27 ++++++++++++++++--------- 5 files changed, 43 insertions(+), 25 deletions(-) diff --git a/builtin/revert.c b/builtin/revert.c index 237f2f18d4c..6165bb10143 100644 --- a/builtin/revert.c +++ b/builtin/revert.c @@ -127,6 +127,8 @@ static int run_sequencer(int argc, const char **argv, struct replay_opts *opts) OPT_BOOL(0, "allow-empty", &opts->allow_empty, N_("preserve initially empty commits")), OPT_BOOL(0, "allow-empty-message", &opts->allow_empty_message, N_("allow commits with empty messages")), OPT_BOOL(0, "keep-redundant-commits", &opts->keep_redundant_commits, N_("keep redundant, empty commits")), + OPT_BOOL_F(0, "rebase-preserve-merges-mode", &opts->rebase_preserve_merges_mode, + N_("use for git-rebase--preserve-merges backend"), PARSE_OPT_HIDDEN), OPT_END(), }; options = parse_options_concat(options, cp_extra); diff --git a/git-rebase--preserve-merges.sh b/git-rebase--preserve-merges.sh index b9c71d2a71b..ca97d9b6539 100644 --- a/git-rebase--preserve-merges.sh +++ b/git-rebase--preserve-merges.sh @@ -444,7 +444,7 @@ pick_one_preserving_merges () { output eval git cherry-pick $allow_rerere_autoupdate \ $allow_empty_message \ ${gpg_sign_opt:+$(git rev-parse --sq-quote "$gpg_sign_opt")} \ - "$strategy_args" "$@" || + "$strategy_args" --rebase-preserve-merges-mode "$@" || die_with_patch $sha1 "$(eval_gettext "Could not pick \$sha1")" ;; esac diff --git a/sequencer.c b/sequencer.c index 0bec01cf38e..ceaf73a34df 100644 --- a/sequencer.c +++ b/sequencer.c @@ -397,13 +397,11 @@ static void free_message(struct commit *commit, struct commit_message *msg) unuse_commit_buffer(commit, msg->message); } -static void print_advice(struct repository *r, int show_hint, - struct replay_opts *opts) +static char *check_need_delete_cherry_pick_head(struct repository *r) { char *msg = getenv("GIT_CHERRY_PICK_HELP"); if (msg) { - fprintf(stderr, "%s\n", msg); /* * A conflict has occurred but the porcelain * (typically rebase --interactive) wants to take care @@ -411,18 +409,22 @@ static void print_advice(struct repository *r, int show_hint, */ refs_delete_ref(get_main_ref_store(r), "", "CHERRY_PICK_HEAD", NULL, 0); - return; + return msg; } + return NULL; +} - if (show_hint) { - if (opts->no_commit) - advise(_("after resolving the conflicts, mark the corrected paths\n" - "with 'git add <paths>' or 'git rm <paths>'")); - else - advise(_("after resolving the conflicts, mark the corrected paths\n" - "with 'git add <paths>' or 'git rm <paths>'\n" - "and commit the result with 'git commit'")); - } +static void print_advice(struct replay_opts *opts, const char *help_msgs) +{ + if (help_msgs) + advise("%s\n", help_msgs); + else if (opts->no_commit) + advise(_("after resolving the conflicts, mark the corrected paths\n" + "with 'git add <paths>' or 'git rm <paths>'")); + else + advise(_("after resolving the conflicts, mark the corrected paths\n" + "with 'git add <paths>' or 'git rm <paths>'\n" + "and commit the result with 'git commit'")); } static int write_message(const void *buf, size_t len, const char *filename, @@ -2261,11 +2263,17 @@ static int do_pick_commit(struct repository *r, res = -1; if (res) { + const char *help_msgs = NULL; + error(command == TODO_REVERT ? _("could not revert %s... %s") : _("could not apply %s... %s"), short_commit_name(commit), msg.subject); - print_advice(r, res == 1, opts); + if (((opts->action == REPLAY_PICK && + !opts->rebase_preserve_merges_mode) || + (help_msgs = check_need_delete_cherry_pick_head(r))) && + res == 1) + print_advice(opts, help_msgs); repo_rerere(r, opts->allow_rerere_auto); goto leave; } diff --git a/sequencer.h b/sequencer.h index d57d8ea23d7..5a40b6d8bdc 100644 --- a/sequencer.h +++ b/sequencer.h @@ -49,6 +49,7 @@ struct replay_opts { int reschedule_failed_exec; int committer_date_is_author_date; int ignore_date; + int rebase_preserve_merges_mode; int mainline; diff --git a/t/t3507-cherry-pick-conflict.sh b/t/t3507-cherry-pick-conflict.sh index 014001b8f32..6f8035399d9 100755 --- a/t/t3507-cherry-pick-conflict.sh +++ b/t/t3507-cherry-pick-conflict.sh @@ -76,6 +76,23 @@ test_expect_success 'advice from failed cherry-pick --no-commit' " test_cmp expected actual " +test_expect_success 'advice from failed cherry-pick with GIT_CHERRY_PICK_HELP' " + pristine_detach initial && + ( + picked=\$(git rev-parse --short picked) && + cat <<-EOF >expected && + error: could not apply \$picked... picked + hint: after resolving the conflicts, mark the corrected paths + hint: with 'git add <paths>' or 'git rm <paths>' + hint: and commit the result with 'git commit' + EOF + GIT_CHERRY_PICK_HELP='and then do something else' && + export GIT_CHERRY_PICK_HELP && + test_must_fail git cherry-pick picked 2>actual && + test_cmp expected actual + ) +" + test_expect_success 'failed cherry-pick sets CHERRY_PICK_HEAD' ' pristine_detach initial && test_must_fail git cherry-pick picked && @@ -109,16 +126,6 @@ test_expect_success \ test_must_fail git rev-parse --verify CHERRY_PICK_HEAD ' -test_expect_success 'GIT_CHERRY_PICK_HELP suppresses CHERRY_PICK_HEAD' ' - pristine_detach initial && - ( - GIT_CHERRY_PICK_HELP="and then do something else" && - export GIT_CHERRY_PICK_HELP && - test_must_fail git cherry-pick picked - ) && - test_must_fail git rev-parse --verify CHERRY_PICK_HEAD -' - test_expect_success 'git reset clears CHERRY_PICK_HEAD' ' pristine_detach initial && -- gitgitgadget ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 1/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP 2021-07-29 12:27 ` [PATCH 1/2] " ZheNing Hu via GitGitGadget @ 2021-07-29 20:56 ` Junio C Hamano 2021-07-30 14:15 ` ZheNing Hu 0 siblings, 1 reply; 8+ messages in thread From: Junio C Hamano @ 2021-07-29 20:56 UTC (permalink / raw) To: ZheNing Hu via GitGitGadget Cc: git, Christian Couder, Hariom Verma, Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys, Ramkumar Ramachandra, Felipe Contreras, Phillip Wood, ZheNing Hu "ZheNing Hu via GitGitGadget" <gitgitgadget@gmail.com> writes: > -static void print_advice(struct repository *r, int show_hint, > - struct replay_opts *opts) > +static char *check_need_delete_cherry_pick_head(struct repository *r) > { > char *msg = getenv("GIT_CHERRY_PICK_HELP"); > > if (msg) { > - fprintf(stderr, "%s\n", msg); > /* > * A conflict has occurred but the porcelain > * (typically rebase --interactive) wants to take care > @@ -411,18 +409,22 @@ static void print_advice(struct repository *r, int show_hint, > */ > refs_delete_ref(get_main_ref_store(r), "", "CHERRY_PICK_HEAD", > NULL, 0); > - return; > + return msg; > } > + return NULL; > +} The patch is a bit hard to read, but I do not think this improves the root cause of the issue at all, which is that "do we need to delete the pseudoref?" decision is tied to and based on the presense of the custom message in the environment variable. The original issue was that we declared (erroneously) that the presence of a custom message _means_ the caller is not a human but a sequencing machinery that wants to deal with what is done via the CHERRY_PICK_HEAD pseudoref itself, and your original issue was that when a human user has the custom message envirnoment (either on purpose or by mistake), the logic _misidentifies_ the call as if it is made from a sequencing machinery and we lose the CHERRY_PICK_HEAD pseudoref. The root cause is mistaking the correlation (sequencing machinery uses the environment and it also does not want pseudoref) as the causation (presense of the environment means we should remove pseudoref). ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP 2021-07-29 20:56 ` Junio C Hamano @ 2021-07-30 14:15 ` ZheNing Hu 0 siblings, 0 replies; 8+ messages in thread From: ZheNing Hu @ 2021-07-30 14:15 UTC (permalink / raw) To: Junio C Hamano Cc: ZheNing Hu via GitGitGadget, Git List, Christian Couder, Hariom Verma, Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys, Ramkumar Ramachandra, Felipe Contreras, Phillip Wood Junio C Hamano <gitster@pobox.com> 于2021年7月30日周五 上午4:56写道: > > The patch is a bit hard to read, but I do not think this improves > the root cause of the issue at all, which is that "do we need to > delete the pseudoref?" decision is tied to and based on the presense > of the custom message in the environment variable. > My original idea was to distinguish between cherry-pick calls by machines (git-rebase--preserve-merges.sh) and cherry-pick calls by people through an additional option "rebase-preserve-merges-mode". > The original issue was that we declared (erroneously) that the > presence of a custom message _means_ the caller is not a human but a > sequencing machinery that wants to deal with what is done via the > CHERRY_PICK_HEAD pseudoref itself, and your original issue was that > when a human user has the custom message envirnoment (either on > purpose or by mistake), the logic _misidentifies_ the call as if it > is made from a sequencing machinery and we lose the CHERRY_PICK_HEAD > pseudoref. The root cause is mistaking the correlation (sequencing > machinery uses the environment and it also does not want pseudoref) > as the causation (presense of the environment means we should remove > pseudoref). Ok... Now I guess what you mean is to provide an option for git cherry-pick to specifically delete "CHERRY_PICK_HEAD", and this options is "hidden", it is not provided to users, it is only reserved for sequencing machinery. In this way, we separate print_advice() from deleting "CHERRY_PICK_HEAD". Then in the future, GIT_CHERRY_PICK_HELP will be deleted. Thanks, -- ZheNing Hu ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 2/2] [GSOC] cherry-pick: use better advice message 2021-07-29 12:27 [PATCH 0/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP ZheNing Hu via GitGitGadget 2021-07-29 12:27 ` [PATCH 1/2] " ZheNing Hu via GitGitGadget @ 2021-07-29 12:27 ` ZheNing Hu via GitGitGadget 1 sibling, 0 replies; 8+ messages in thread From: ZheNing Hu via GitGitGadget @ 2021-07-29 12:27 UTC (permalink / raw) To: git Cc: Junio C Hamano, Christian Couder, Hariom Verma, Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys, Ramkumar Ramachandra, Felipe Contreras, Phillip Wood, ZheNing Hu, ZheNing Hu From: ZheNing Hu <adlternative@gmail.com> In the past, git cherry-pick would print such advice when there was a conflict: hint: after resolving the conflicts, mark the corrected paths hint: with 'git add <paths>' or 'git rm <paths>' hint: and commit the result with 'git commit' But in fact, when we want to cherry-pick multiple commits, we should not use "git commit" after resolving conflicts, which will make Git generate some errors. We should recommend users to use `git cherry-pick --continue`, `git cherry-pick --abort`, just like git rebase does. This is the improved advice: hint: Resolve all conflicts manually, mark them as resolved with hint: "git add/rm <conflicted_files>", then run "git cherry-pick \ --continue". hint: You can instead skip this commit: run "git cherry-pick --skip". hint: To abort and get back to the state before "git cherry-pick", hint: run "git cherry-pick --abort". Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by Hariom Verma <hariom18599@gmail.com>: Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk> Hepled-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: ZheNing Hu <adlternative@gmail.com> --- sequencer.c | 19 ++++++++++++++----- t/t3507-cherry-pick-conflict.sh | 23 +++++++++++++++-------- 2 files changed, 29 insertions(+), 13 deletions(-) diff --git a/sequencer.c b/sequencer.c index ceaf73a34df..b9adeb7e5f5 100644 --- a/sequencer.c +++ b/sequencer.c @@ -39,6 +39,16 @@ static const char sign_off_header[] = "Signed-off-by: "; static const char cherry_picked_prefix[] = "(cherry picked from commit "; +static const char *no_commit_advice = N_("after resolving the conflicts, mark the corrected paths\n" + "with 'git add <paths>' or 'git rm <paths>'"); +static const char *commit_advice = N_("after resolving the conflicts, mark the corrected paths\n" + "with 'git add <paths>' or 'git rm <paths>'\n" + "and commit the result with 'git commit'"); +static const char *cherry_pick_advice = N_("Resolve all conflicts manually, mark them as resolved with\n" + "\"git add/rm <conflicted_files>\", then run \"git cherry-pick --continue\".\n" + "You can instead skip this commit: run \"git cherry-pick --skip\".\n" + "To abort and get back to the state before \"git cherry-pick\",\n" + "run \"git cherry-pick --abort\"."); GIT_PATH_FUNC(git_path_commit_editmsg, "COMMIT_EDITMSG") @@ -419,12 +429,9 @@ static void print_advice(struct replay_opts *opts, const char *help_msgs) if (help_msgs) advise("%s\n", help_msgs); else if (opts->no_commit) - advise(_("after resolving the conflicts, mark the corrected paths\n" - "with 'git add <paths>' or 'git rm <paths>'")); + advise("%s\n", _(no_commit_advice)); else - advise(_("after resolving the conflicts, mark the corrected paths\n" - "with 'git add <paths>' or 'git rm <paths>'\n" - "and commit the result with 'git commit'")); + advise("%s\n", _(commit_advice)); } static int write_message(const void *buf, size_t len, const char *filename, @@ -2269,6 +2276,8 @@ static int do_pick_commit(struct repository *r, ? _("could not revert %s... %s") : _("could not apply %s... %s"), short_commit_name(commit), msg.subject); + if (opts->action == REPLAY_PICK) + help_msgs = _(cherry_pick_advice); if (((opts->action == REPLAY_PICK && !opts->rebase_preserve_merges_mode) || (help_msgs = check_need_delete_cherry_pick_head(r))) && diff --git a/t/t3507-cherry-pick-conflict.sh b/t/t3507-cherry-pick-conflict.sh index 6f8035399d9..bf77bb0bd50 100755 --- a/t/t3507-cherry-pick-conflict.sh +++ b/t/t3507-cherry-pick-conflict.sh @@ -53,9 +53,11 @@ test_expect_success 'advice from failed cherry-pick' " picked=\$(git rev-parse --short picked) && cat <<-EOF >expected && error: could not apply \$picked... picked - hint: after resolving the conflicts, mark the corrected paths - hint: with 'git add <paths>' or 'git rm <paths>' - hint: and commit the result with 'git commit' + hint: Resolve all conflicts manually, mark them as resolved with + hint: \"git add/rm <conflicted_files>\", then run \"git cherry-pick --continue\". + hint: You can instead skip this commit: run \"git cherry-pick --skip\". + hint: To abort and get back to the state before \"git cherry-pick\", + hint: run \"git cherry-pick --abort\". EOF test_must_fail git cherry-pick picked 2>actual && @@ -68,8 +70,11 @@ test_expect_success 'advice from failed cherry-pick --no-commit' " picked=\$(git rev-parse --short picked) && cat <<-EOF >expected && error: could not apply \$picked... picked - hint: after resolving the conflicts, mark the corrected paths - hint: with 'git add <paths>' or 'git rm <paths>' + hint: Resolve all conflicts manually, mark them as resolved with + hint: \"git add/rm <conflicted_files>\", then run \"git cherry-pick --continue\". + hint: You can instead skip this commit: run \"git cherry-pick --skip\". + hint: To abort and get back to the state before \"git cherry-pick\", + hint: run \"git cherry-pick --abort\". EOF test_must_fail git cherry-pick --no-commit picked 2>actual && @@ -82,9 +87,11 @@ test_expect_success 'advice from failed cherry-pick with GIT_CHERRY_PICK_HELP' " picked=\$(git rev-parse --short picked) && cat <<-EOF >expected && error: could not apply \$picked... picked - hint: after resolving the conflicts, mark the corrected paths - hint: with 'git add <paths>' or 'git rm <paths>' - hint: and commit the result with 'git commit' + hint: Resolve all conflicts manually, mark them as resolved with + hint: \"git add/rm <conflicted_files>\", then run \"git cherry-pick --continue\". + hint: You can instead skip this commit: run \"git cherry-pick --skip\". + hint: To abort and get back to the state before \"git cherry-pick\", + hint: run \"git cherry-pick --abort\". EOF GIT_CHERRY_PICK_HELP='and then do something else' && export GIT_CHERRY_PICK_HELP && -- gitgitgadget ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 0/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP @ 2021-07-31 7:01 ZheNing Hu via GitGitGadget 2021-07-31 7:01 ` [PATCH 1/2] " ZheNing Hu via GitGitGadget 0 siblings, 1 reply; 8+ messages in thread From: ZheNing Hu via GitGitGadget @ 2021-07-31 7:01 UTC (permalink / raw) To: git Cc: Junio C Hamano, Christian Couder, Hariom Verma, Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys, Ramkumar Ramachandra, Felipe Contreras, Phillip Wood, ZheNing Hu This patch fixes the bug when git cherry-pick is used with environment variable GIT_CHERRY_PICK_HELP, and makes git chery-pick advice message better. v3: https://lore.kernel.org/git/pull.1007.git.1627561665.gitgitgadget@gmail.com/ v3-->v4: 1. Add hidden option --delete-cherry-pick-head to git cherry-pick wihch used to delete CHERRY_PICK_HEAD when conflict occurs. 2. add delete_cherry_pick_head flag to struct replay_opts and struct rebase_options. 3. Split print_advice() into print advice and delete CHERRY_PICK_HEAD two part. 4. Use better git cherry-pick advice message. ZheNing Hu (2): [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP [GSOC] cherry-pick: use better advice message builtin/rebase.c | 3 +++ builtin/revert.c | 2 ++ git-rebase--preserve-merges.sh | 2 +- sequencer.c | 37 +++++++++++++++++------------- sequencer.h | 1 + t/t3507-cherry-pick-conflict.sh | 40 ++++++++++++++++++++------------- 6 files changed, 53 insertions(+), 32 deletions(-) base-commit: daab8a564f8bbac55f70f8bf86c070e001a9b006 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1010%2Fadlternative%2Fcherry-pick-help-fix-3-v1 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1010/adlternative/cherry-pick-help-fix-3-v1 Pull-Request: https://github.com/gitgitgadget/git/pull/1010 -- gitgitgadget ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 1/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP 2021-07-31 7:01 [PATCH 0/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP ZheNing Hu via GitGitGadget @ 2021-07-31 7:01 ` ZheNing Hu via GitGitGadget 2021-08-01 10:09 ` Phillip Wood 0 siblings, 1 reply; 8+ messages in thread From: ZheNing Hu via GitGitGadget @ 2021-07-31 7:01 UTC (permalink / raw) To: git Cc: Junio C Hamano, Christian Couder, Hariom Verma, Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys, Ramkumar Ramachandra, Felipe Contreras, Phillip Wood, ZheNing Hu, ZheNing Hu From: ZheNing Hu <adlternative@gmail.com> GIT_CHERRY_PICK_HELP is an environment variable, as the implementation detail of some porcelain in git to help realize the rebasing steps. E.g. `git rebase -p` set GIT_CHERRY_PICK_HELP value in `git-rebase--preserve-merges.sh`, `git rebase --merge` set GIT_CHERRY_PICK_HELP value in run_specific_rebase(). But If we set the value of GIT_CHERRY_PICK_HELP when using `git cherry-pick`, CHERRY_PICK_HEAD will be deleted, then we will get an error when we try to use `git cherry-pick --continue` or other cherr-pick command. Introduce new "hidden" option `--delete-cherry-pick-head` for git cherry-pick which indicates that CHERRY_PICK_HEAD will be deleted when conflict occurs, which provided for some porcelain commands of git like `git-rebase--preserve-merges.sh`. After `git rebase -p` completely abolished, this option should be removed. At the same time, add the flag `delete_cherry_pick_head` to `struct rebase_options` and `struct replay_opts`, We can decide whether to delete CHERRY_PICK_HEAD by setting, passing, and checking this flag bit. Then we split print_advice() into two part: Firstly, print_advice() will only be responsible for outputting content; Secondly, check if we set the `delete_cherry_pick_head` flag; if set, delete CHERRY_PICK_HEAD. In this way, the steps of printing advice and deleting CHERRY_PICK_HEAD are decoupled. Finally, let `git-rebase--preserve-merges.sh` use the `--delete-cherry-pick-head` option when it executes git cherry-pick, and set the `delete_cherry_pick_head` flag in run_specific_rebase() when we are using `git rebase --merge`, which can fix this breakage. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by Hariom Verma <hariom18599@gmail.com>: Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk> Hepled-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: ZheNing Hu <adlternative@gmail.com> --- builtin/rebase.c | 3 +++ builtin/revert.c | 2 ++ git-rebase--preserve-merges.sh | 2 +- sequencer.c | 28 +++++++++++++--------------- sequencer.h | 1 + t/t3507-cherry-pick-conflict.sh | 25 +++++++++++++++---------- 6 files changed, 35 insertions(+), 26 deletions(-) diff --git a/builtin/rebase.c b/builtin/rebase.c index 12f093121d9..5983f37d531 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -84,6 +84,7 @@ struct rebase_options { REBASE_FORCE = 1<<3, REBASE_INTERACTIVE_EXPLICIT = 1<<4, } flags; + int delete_cherry_pick_head; struct strvec git_am_opts; const char *action; int signoff; @@ -152,6 +153,7 @@ static struct replay_opts get_replay_opts(const struct rebase_options *opts) oidcpy(&replay.squash_onto, opts->squash_onto); replay.have_squash_onto = 1; } + replay.delete_cherry_pick_head = opts->delete_cherry_pick_head; return replay; } @@ -948,6 +950,7 @@ static int run_specific_rebase(struct rebase_options *opts, enum action action) if (opts->type == REBASE_MERGE) { /* Run sequencer-based rebase */ setenv("GIT_CHERRY_PICK_HELP", resolvemsg, 1); + opts->delete_cherry_pick_head = 1; if (!(opts->flags & REBASE_INTERACTIVE_EXPLICIT)) { setenv("GIT_SEQUENCE_EDITOR", ":", 1); opts->autosquash = 0; diff --git a/builtin/revert.c b/builtin/revert.c index 237f2f18d4c..15a4b6fe4ee 100644 --- a/builtin/revert.c +++ b/builtin/revert.c @@ -127,6 +127,8 @@ static int run_sequencer(int argc, const char **argv, struct replay_opts *opts) OPT_BOOL(0, "allow-empty", &opts->allow_empty, N_("preserve initially empty commits")), OPT_BOOL(0, "allow-empty-message", &opts->allow_empty_message, N_("allow commits with empty messages")), OPT_BOOL(0, "keep-redundant-commits", &opts->keep_redundant_commits, N_("keep redundant, empty commits")), + OPT_BOOL_F(0, "delete-cherry-pick-head", &opts->delete_cherry_pick_head, + N_("delete CHERRY_PICK_HEAD when conflict occurs"), PARSE_OPT_HIDDEN), OPT_END(), }; options = parse_options_concat(options, cp_extra); diff --git a/git-rebase--preserve-merges.sh b/git-rebase--preserve-merges.sh index b9c71d2a71b..eaa8f9de2c5 100644 --- a/git-rebase--preserve-merges.sh +++ b/git-rebase--preserve-merges.sh @@ -444,7 +444,7 @@ pick_one_preserving_merges () { output eval git cherry-pick $allow_rerere_autoupdate \ $allow_empty_message \ ${gpg_sign_opt:+$(git rev-parse --sq-quote "$gpg_sign_opt")} \ - "$strategy_args" "$@" || + "$strategy_args" --delete-cherry-pick-head "$@" || die_with_patch $sha1 "$(eval_gettext "Could not pick \$sha1")" ;; esac diff --git a/sequencer.c b/sequencer.c index 0bec01cf38e..83cf6a5da3c 100644 --- a/sequencer.c +++ b/sequencer.c @@ -397,24 +397,13 @@ static void free_message(struct commit *commit, struct commit_message *msg) unuse_commit_buffer(commit, msg->message); } -static void print_advice(struct repository *r, int show_hint, - struct replay_opts *opts) +static void print_advice(struct replay_opts *opts, int show_hint) { char *msg = getenv("GIT_CHERRY_PICK_HELP"); if (msg) { - fprintf(stderr, "%s\n", msg); - /* - * A conflict has occurred but the porcelain - * (typically rebase --interactive) wants to take care - * of the commit itself so remove CHERRY_PICK_HEAD - */ - refs_delete_ref(get_main_ref_store(r), "", "CHERRY_PICK_HEAD", - NULL, 0); - return; - } - - if (show_hint) { + advise("%s\n", msg); + } else if (show_hint) { if (opts->no_commit) advise(_("after resolving the conflicts, mark the corrected paths\n" "with 'git add <paths>' or 'git rm <paths>'")); @@ -2265,7 +2254,16 @@ static int do_pick_commit(struct repository *r, ? _("could not revert %s... %s") : _("could not apply %s... %s"), short_commit_name(commit), msg.subject); - print_advice(r, res == 1, opts); + print_advice(opts, res == 1); + if (opts->delete_cherry_pick_head) { + /* + * A conflict has occurred but the porcelain + * (typically rebase --interactive) wants to take care + * of the commit itself so remove CHERRY_PICK_HEAD + */ + refs_delete_ref(get_main_ref_store(r), "", "CHERRY_PICK_HEAD", + NULL, 0); + } repo_rerere(r, opts->allow_rerere_auto); goto leave; } diff --git a/sequencer.h b/sequencer.h index d57d8ea23d7..76fb4af56fd 100644 --- a/sequencer.h +++ b/sequencer.h @@ -49,6 +49,7 @@ struct replay_opts { int reschedule_failed_exec; int committer_date_is_author_date; int ignore_date; + int delete_cherry_pick_head; int mainline; diff --git a/t/t3507-cherry-pick-conflict.sh b/t/t3507-cherry-pick-conflict.sh index 014001b8f32..f17621d1915 100755 --- a/t/t3507-cherry-pick-conflict.sh +++ b/t/t3507-cherry-pick-conflict.sh @@ -76,6 +76,21 @@ test_expect_success 'advice from failed cherry-pick --no-commit' " test_cmp expected actual " +test_expect_success 'advice from failed cherry-pick with GIT_CHERRY_PICK_HELP' " + pristine_detach initial && + ( + picked=\$(git rev-parse --short picked) && + cat <<-EOF >expected && + error: could not apply \$picked... picked + hint: and then do something else + EOF + GIT_CHERRY_PICK_HELP='and then do something else' && + export GIT_CHERRY_PICK_HELP && + test_must_fail git cherry-pick picked 2>actual && + test_cmp expected actual + ) +" + test_expect_success 'failed cherry-pick sets CHERRY_PICK_HEAD' ' pristine_detach initial && test_must_fail git cherry-pick picked && @@ -109,16 +124,6 @@ test_expect_success \ test_must_fail git rev-parse --verify CHERRY_PICK_HEAD ' -test_expect_success 'GIT_CHERRY_PICK_HELP suppresses CHERRY_PICK_HEAD' ' - pristine_detach initial && - ( - GIT_CHERRY_PICK_HELP="and then do something else" && - export GIT_CHERRY_PICK_HELP && - test_must_fail git cherry-pick picked - ) && - test_must_fail git rev-parse --verify CHERRY_PICK_HEAD -' - test_expect_success 'git reset clears CHERRY_PICK_HEAD' ' pristine_detach initial && -- gitgitgadget ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 1/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP 2021-07-31 7:01 ` [PATCH 1/2] " ZheNing Hu via GitGitGadget @ 2021-08-01 10:09 ` Phillip Wood 2021-08-02 13:34 ` ZheNing Hu 0 siblings, 1 reply; 8+ messages in thread From: Phillip Wood @ 2021-08-01 10:09 UTC (permalink / raw) To: ZheNing Hu via GitGitGadget, git Cc: Junio C Hamano, Christian Couder, Hariom Verma, Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys, Ramkumar Ramachandra, Felipe Contreras, ZheNing Hu On 31/07/2021 08:01, ZheNing Hu via GitGitGadget wrote: > From: ZheNing Hu <adlternative@gmail.com> > > GIT_CHERRY_PICK_HELP is an environment variable, as the > implementation detail of some porcelain in git to help realize > the rebasing steps. E.g. `git rebase -p` set GIT_CHERRY_PICK_HELP > value in `git-rebase--preserve-merges.sh`, `git rebase --merge` set > GIT_CHERRY_PICK_HELP value in run_specific_rebase(). But If we set > the value of GIT_CHERRY_PICK_HELP when using `git cherry-pick`, > CHERRY_PICK_HEAD will be deleted, then we will get an error when we > try to use `git cherry-pick --continue` or other cherr-pick command. > > Introduce new "hidden" option `--delete-cherry-pick-head` for git > cherry-pick which indicates that CHERRY_PICK_HEAD will be deleted when > conflict occurs, which provided for some porcelain commands of git like > `git-rebase--preserve-merges.sh`. After `git rebase -p` completely > abolished, this option should be removed. At the same time, add the flag > `delete_cherry_pick_head` to `struct rebase_options` and > `struct replay_opts`, We can decide whether to delete CHERRY_PICK_HEAD > by setting, passing, and checking this flag bit. > > Then we split print_advice() into two part: Firstly, print_advice() > will only be responsible for outputting content; Secondly, check if > we set the `delete_cherry_pick_head` flag; if set, delete CHERRY_PICK_HEAD. > In this way, the steps of printing advice and deleting CHERRY_PICK_HEAD > are decoupled. Finally, let `git-rebase--preserve-merges.sh` use the > `--delete-cherry-pick-head` option when it executes git cherry-pick, and > set the `delete_cherry_pick_head` flag in run_specific_rebase() when we > are using `git rebase --merge`, which can fix this breakage. > > Mentored-by: Christian Couder <christian.couder@gmail.com> > Mentored-by Hariom Verma <hariom18599@gmail.com>: > Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk> > Hepled-by: Junio C Hamano <gitster@pobox.com> > Signed-off-by: ZheNing Hu <adlternative@gmail.com> > --- > builtin/rebase.c | 3 +++ > builtin/revert.c | 2 ++ > git-rebase--preserve-merges.sh | 2 +- > sequencer.c | 28 +++++++++++++--------------- > sequencer.h | 1 + > t/t3507-cherry-pick-conflict.sh | 25 +++++++++++++++---------- > 6 files changed, 35 insertions(+), 26 deletions(-) > > diff --git a/builtin/rebase.c b/builtin/rebase.c > index 12f093121d9..5983f37d531 100644 > --- a/builtin/rebase.c > +++ b/builtin/rebase.c > @@ -84,6 +84,7 @@ struct rebase_options { > REBASE_FORCE = 1<<3, > REBASE_INTERACTIVE_EXPLICIT = 1<<4, > } flags; > + int delete_cherry_pick_head; > struct strvec git_am_opts; > const char *action; > int signoff; > @@ -152,6 +153,7 @@ static struct replay_opts get_replay_opts(const struct rebase_options *opts) > oidcpy(&replay.squash_onto, opts->squash_onto); > replay.have_squash_onto = 1; > } > + replay.delete_cherry_pick_head = opts->delete_cherry_pick_head; I think we could just have replay.delete_cherry_pick_head = 1; and not add a new member to rebase_options as we always want this set > > return replay; > } > @@ -948,6 +950,7 @@ static int run_specific_rebase(struct rebase_options *opts, enum action action) > if (opts->type == REBASE_MERGE) { > /* Run sequencer-based rebase */ > setenv("GIT_CHERRY_PICK_HELP", resolvemsg, 1); > + opts->delete_cherry_pick_head = 1; > if (!(opts->flags & REBASE_INTERACTIVE_EXPLICIT)) { > setenv("GIT_SEQUENCE_EDITOR", ":", 1); > opts->autosquash = 0; > diff --git a/builtin/revert.c b/builtin/revert.c > index 237f2f18d4c..15a4b6fe4ee 100644 > --- a/builtin/revert.c > +++ b/builtin/revert.c > @@ -127,6 +127,8 @@ static int run_sequencer(int argc, const char **argv, struct replay_opts *opts) > OPT_BOOL(0, "allow-empty", &opts->allow_empty, N_("preserve initially empty commits")), > OPT_BOOL(0, "allow-empty-message", &opts->allow_empty_message, N_("allow commits with empty messages")), > OPT_BOOL(0, "keep-redundant-commits", &opts->keep_redundant_commits, N_("keep redundant, empty commits")), > + OPT_BOOL_F(0, "delete-cherry-pick-head", &opts->delete_cherry_pick_head, > + N_("delete CHERRY_PICK_HEAD when conflict occurs"), PARSE_OPT_HIDDEN), > OPT_END(), > }; > options = parse_options_concat(options, cp_extra); > diff --git a/git-rebase--preserve-merges.sh b/git-rebase--preserve-merges.sh > index b9c71d2a71b..eaa8f9de2c5 100644 > --- a/git-rebase--preserve-merges.sh > +++ b/git-rebase--preserve-merges.sh > @@ -444,7 +444,7 @@ pick_one_preserving_merges () { > output eval git cherry-pick $allow_rerere_autoupdate \ > $allow_empty_message \ > ${gpg_sign_opt:+$(git rev-parse --sq-quote "$gpg_sign_opt")} \ > - "$strategy_args" "$@" || > + "$strategy_args" --delete-cherry-pick-head "$@" || > die_with_patch $sha1 "$(eval_gettext "Could not pick \$sha1")" > ;; > esac > diff --git a/sequencer.c b/sequencer.c > index 0bec01cf38e..83cf6a5da3c 100644 > --- a/sequencer.c > +++ b/sequencer.c > @@ -397,24 +397,13 @@ static void free_message(struct commit *commit, struct commit_message *msg) > unuse_commit_buffer(commit, msg->message); > } > > -static void print_advice(struct repository *r, int show_hint, > - struct replay_opts *opts) > +static void print_advice(struct replay_opts *opts, int show_hint) > { > char *msg = getenv("GIT_CHERRY_PICK_HELP"); > > if (msg) { > - fprintf(stderr, "%s\n", msg); > - /* > - * A conflict has occurred but the porcelain > - * (typically rebase --interactive) wants to take care > - * of the commit itself so remove CHERRY_PICK_HEAD > - */ > - refs_delete_ref(get_main_ref_store(r), "", "CHERRY_PICK_HEAD", > - NULL, 0); > - return; > - } > - > - if (show_hint) { > + advise("%s\n", msg); This is a change in behavior as the advice will now be prefixed with "hint: ". I think that is probably an improvement so long as it does not make the lines too long when the advice is printed. > + } else if (show_hint) { > if (opts->no_commit) > advise(_("after resolving the conflicts, mark the corrected paths\n" > "with 'git add <paths>' or 'git rm <paths>'")); > @@ -2265,7 +2254,16 @@ static int do_pick_commit(struct repository *r, > ? _("could not revert %s... %s") > : _("could not apply %s... %s"), > short_commit_name(commit), msg.subject); > - print_advice(r, res == 1, opts); > + print_advice(opts, res == 1); > + if (opts->delete_cherry_pick_head) { > + /* > + * A conflict has occurred but the porcelain > + * (typically rebase --interactive) wants to take care > + * of the commit itself so remove CHERRY_PICK_HEAD > + */ > + refs_delete_ref(get_main_ref_store(r), "", "CHERRY_PICK_HEAD", > + NULL, 0); > + } > repo_rerere(r, opts->allow_rerere_auto); > goto leave; > } > diff --git a/sequencer.h b/sequencer.h > index d57d8ea23d7..76fb4af56fd 100644 > --- a/sequencer.h > +++ b/sequencer.h > @@ -49,6 +49,7 @@ struct replay_opts { > int reschedule_failed_exec; > int committer_date_is_author_date; > int ignore_date; > + int delete_cherry_pick_head; > > int mainline; > > diff --git a/t/t3507-cherry-pick-conflict.sh b/t/t3507-cherry-pick-conflict.sh > index 014001b8f32..f17621d1915 100755 > --- a/t/t3507-cherry-pick-conflict.sh > +++ b/t/t3507-cherry-pick-conflict.sh > @@ -76,6 +76,21 @@ test_expect_success 'advice from failed cherry-pick --no-commit' " > test_cmp expected actual > " > > +test_expect_success 'advice from failed cherry-pick with GIT_CHERRY_PICK_HELP' " > + pristine_detach initial && > + ( > + picked=\$(git rev-parse --short picked) && > + cat <<-EOF >expected && > + error: could not apply \$picked... picked > + hint: and then do something else > + EOF > + GIT_CHERRY_PICK_HELP='and then do something else' && > + export GIT_CHERRY_PICK_HELP && > + test_must_fail git cherry-pick picked 2>actual && > + test_cmp expected actual > + ) > +" > + > test_expect_success 'failed cherry-pick sets CHERRY_PICK_HEAD' ' > pristine_detach initial && > test_must_fail git cherry-pick picked && > @@ -109,16 +124,6 @@ test_expect_success \ > test_must_fail git rev-parse --verify CHERRY_PICK_HEAD > ' > > -test_expect_success 'GIT_CHERRY_PICK_HELP suppresses CHERRY_PICK_HEAD' ' > - pristine_detach initial && > - ( > - GIT_CHERRY_PICK_HELP="and then do something else" && > - export GIT_CHERRY_PICK_HELP && > - test_must_fail git cherry-pick picked > - ) && > - test_must_fail git rev-parse --verify CHERRY_PICK_HEAD > -' > - > test_expect_success 'git reset clears CHERRY_PICK_HEAD' ' > pristine_detach initial && I think it would be useful to add a test for the new cherry-pick option that is added in this patch. Overall this patch is looking pretty good it just needs a few small changes, thanks for working on it. Best Wishes Phillip ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP 2021-08-01 10:09 ` Phillip Wood @ 2021-08-02 13:34 ` ZheNing Hu 0 siblings, 0 replies; 8+ messages in thread From: ZheNing Hu @ 2021-08-02 13:34 UTC (permalink / raw) To: phillip.wood Cc: ZheNing Hu via GitGitGadget, Git List, Junio C Hamano, Christian Couder, Hariom Verma, Ævar Arnfjörð Bjarmason, Han-Wen Nienhuys, Ramkumar Ramachandra, Felipe Contreras Phillip Wood <phillip.wood123@gmail.com> 于2021年8月1日周日 下午6:10写道: > > > diff --git a/builtin/rebase.c b/builtin/rebase.c > > index 12f093121d9..5983f37d531 100644 > > --- a/builtin/rebase.c > > +++ b/builtin/rebase.c > > @@ -84,6 +84,7 @@ struct rebase_options { > > REBASE_FORCE = 1<<3, > > REBASE_INTERACTIVE_EXPLICIT = 1<<4, > > } flags; > > + int delete_cherry_pick_head; > > struct strvec git_am_opts; > > const char *action; > > int signoff; > > @@ -152,6 +153,7 @@ static struct replay_opts get_replay_opts(const struct rebase_options *opts) > > oidcpy(&replay.squash_onto, opts->squash_onto); > > replay.have_squash_onto = 1; > > } > > + replay.delete_cherry_pick_head = opts->delete_cherry_pick_head; > > I think we could just have > replay.delete_cherry_pick_head = 1; > and not add a new member to rebase_options as we always want this set Maybe you are right. > > -static void print_advice(struct repository *r, int show_hint, > > - struct replay_opts *opts) > > +static void print_advice(struct replay_opts *opts, int show_hint) > > { > > char *msg = getenv("GIT_CHERRY_PICK_HELP"); > > > > if (msg) { > > - fprintf(stderr, "%s\n", msg); > > - /* > > - * A conflict has occurred but the porcelain > > - * (typically rebase --interactive) wants to take care > > - * of the commit itself so remove CHERRY_PICK_HEAD > > - */ > > - refs_delete_ref(get_main_ref_store(r), "", "CHERRY_PICK_HEAD", > > - NULL, 0); > > - return; > > - } > > - > > - if (show_hint) { > > + advise("%s\n", msg); > > This is a change in behavior as the advice will now be prefixed with > "hint: ". I think that is probably an improvement so long as it does not > make the lines too long when the advice is printed. > Yeah, maybe I should mention this in commit messages. > > I think it would be useful to add a test for the new cherry-pick option > that is added in this patch. Overall this patch is looking pretty good > it just needs a few small changes, thanks for working on it. > Make sence. > Best Wishes > > Phillip Thanks. -- ZheNing Hu ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-08-02 13:36 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-07-29 12:27 [PATCH 0/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP ZheNing Hu via GitGitGadget 2021-07-29 12:27 ` [PATCH 1/2] " ZheNing Hu via GitGitGadget 2021-07-29 20:56 ` Junio C Hamano 2021-07-30 14:15 ` ZheNing Hu 2021-07-29 12:27 ` [PATCH 2/2] [GSOC] cherry-pick: use better advice message ZheNing Hu via GitGitGadget 2021-07-31 7:01 [PATCH 0/2] [GSOC] cherry-pick: fix bug when used with GIT_CHERRY_PICK_HELP ZheNing Hu via GitGitGadget 2021-07-31 7:01 ` [PATCH 1/2] " ZheNing Hu via GitGitGadget 2021-08-01 10:09 ` Phillip Wood 2021-08-02 13:34 ` ZheNing Hu
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.