Re: [PATCH 8/9] update-index: integrate with sparse index

[Elijah Newren <newren@gmail.com>]

On Mon, Jan 10, 2022 at 6:10 AM Victoria Dye <vdye@github.com> wrote:
>
> Elijah Newren wrote:
> > On Tue, Jan 4, 2022 at 9:37 AM Victoria Dye via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:
> >>
> >> From: Victoria Dye <vdye@github.com>
> >>
> >> Enable usage of the sparse index with `update-index`. Most variations of
> >> `update-index` work without explicitly expanding the index or making any
> >> other updates in or outside of `update-index.c`.
> >>
> >> The one usage requiring additional changes is `--cacheinfo`; if a file
> >> inside a sparse directory was specified, the index would not be expanded
> >> until after the cache tree is invalidated, leading to a mismatch between the
> >> index and cache tree. This scenario is handled by rearranging
> >> `add_index_entry_with_check`, allowing `index_name_stage_pos` to expand the
> >> index *before* attempting to invalidate the relevant cache tree path,
> >> avoiding cache tree/index corruption.
> >>
> >> Signed-off-by: Victoria Dye <vdye@github.com>
> >> ---
> >>  builtin/update-index.c                   |  3 +++
> >>  read-cache.c                             | 10 +++++++---
> >>  t/t1092-sparse-checkout-compatibility.sh | 12 ++++++++++++
> >>  3 files changed, 22 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/builtin/update-index.c b/builtin/update-index.c
> >> index 187203e8bb5..605cc693bbd 100644
> >> --- a/builtin/update-index.c
> >> +++ b/builtin/update-index.c
> >> @@ -1077,6 +1077,9 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
> >>
> >>         git_config(git_default_config, NULL);
> >>
> >> +       prepare_repo_settings(r);
> >> +       the_repository->settings.command_requires_full_index = 0;
> >> +
> >>         /* we will diagnose later if it turns out that we need to update it */
> >>         newfd = hold_locked_index(&lock_file, 0);
> >>         if (newfd < 0)
> >> diff --git a/read-cache.c b/read-cache.c
> >> index cbe73f14e5e..b4600e954b6 100644
> >> --- a/read-cache.c
> >> +++ b/read-cache.c
> >> @@ -1339,9 +1339,6 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e
> >>         int skip_df_check = option & ADD_CACHE_SKIP_DFCHECK;
> >>         int new_only = option & ADD_CACHE_NEW_ONLY;
> >>
> >> -       if (!(option & ADD_CACHE_KEEP_CACHE_TREE))
> >> -               cache_tree_invalidate_path(istate, ce->name);
> >> -
> >>         /*
> >>          * If this entry's path sorts after the last entry in the index,
> >>          * we can avoid searching for it.
> >> @@ -1352,6 +1349,13 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e
> >>         else
> >>                 pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE);
> >>
> >> +       /*
> >> +        * Cache tree path should be invalidated only after index_name_stage_pos,
> >> +        * in case it expands a sparse index.
> >> +        */
> >> +       if (!(option & ADD_CACHE_KEEP_CACHE_TREE))
> >> +               cache_tree_invalidate_path(istate, ce->name);
> >> +
> >>         /* existing match? Just replace it. */
> >>         if (pos >= 0) {
> >>                 if (!new_only)
> >> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh
> >> index 6804ab23a27..bc0741c970d 100755
> >> --- a/t/t1092-sparse-checkout-compatibility.sh
> >> +++ b/t/t1092-sparse-checkout-compatibility.sh
> >> @@ -1216,6 +1216,18 @@ test_expect_success 'sparse index is not expanded: blame' '
> >>         done
> >>  '
> >>
> >> +test_expect_success 'sparse index is not expanded: update-index' '
> >> +       init_repos &&
> >> +
> >> +       echo "test" >sparse-index/README.md &&
> >> +       echo "test2" >sparse-index/a &&
> >> +       rm -f sparse-index/deep/a &&
> >> +
> >> +       ensure_not_expanded update-index --add README.md &&
> >> +       ensure_not_expanded update-index a &&
> >> +       ensure_not_expanded update-index --remove deep/a
> >> +'
> >
> > The commit message said this change was about --cacheinfo, but this
> > test doesn't use that option.  I'm confused; was this a bad patch
> > splitting by chance?
> >
>
> It was not - the commit message title is "update-index: integrate with
> sparse index" and the message starts by saying that this patch enables use
> of the sparse index for *all* of `update-index` (where "[m]ost variations of
> `update-index` work without...making any other updates in or outside of
> `update-index.c`").
>
> It goes on to say that the `--cache-info` option is an exception to the
> above statement (that most variations work without updates outside
> `update-index.c`) because it requires a slight change to
> `add_index_entry_with_check(...)` to avoid index corruption. That change is
> also in this patch, but it's not the main "focus".
>
> The test added here is intended to broadly test `update-index` with a sparse
> index. I'll add a case for `--cacheinfo` in my next re-roll but, for
> context, the reason I didn't originally is because I focused on the (as far
> as I could tell) most commonly-used variations of `update-index`.

Ah, I see.  I didn't read closely enough and then saw the majority of
the commit message and code changes were about --cacheinfo, but the
testcase wasn't.  Sorry about that.

> >> +
> >>  # NEEDSWORK: a sparse-checkout behaves differently from a full checkout
> >>  # in this scenario, but it shouldn't.
> >>  test_expect_success 'reset mixed and checkout orphan' '
> >> --
> >> gitgitgadget
> >>